Baby Analysis with Musings

Created	@September 23, 2022 3:08 PM
Last Edited by	TThe Grand Inquisitor
Last Edited Time	@February 3, 2023 10:34 AM

Index for Homework and Quizzes

Index	HW 1	HW 2	HW 3	HW 4	Qz 1	Qz 3	Eamples
1	See/	See/	See	See	1,2	See	See
2	See/	See	See	See			See*
3	See/	See	1/,2,3	See	See		See
4	See	1, 2,3 , 4	See	See			See
5	2,3,4	See	See	See			See
6			See				See
7			See				See
8			1, 2				See
9			See				See
10			1,2,3,4				See
11			1,2				See
12			See

0. 驯龙集

$\varepsilon$ 变脸法：缝朝服，至公卿 (take $\varepsilon=...$ 的操作)
- Prove: UF-1 $\Rarr$ Standard definition of $\lim_{n\rarr\infty}a_n= l$ .
  By UF-1, $\forall w>0,\exists N(w),s.t.~|a_n-l|<cw,\forall n\ge N(w).$ Now take $w=\frac\varepsilon{2c}$ , and standard defn holds.
- Prove of Fact 1.5
  $\because n_k\rarr\infty\text{ as }k\rarr\infin\therefore \exists K_\varepsilon~s.t.~n_k\ge N_\varepsilon\text{ as long as }k\ge K_\varepsilon.\because \lim a_n=l\therefore |a_{n_k}-l|<\varepsilon,\forall k\ge K_\varepsilon\Rarr \lim a_{n_k}=l.\blacksquare$
- Proof of Fact 2.3.1
- Hw1 T3 Back to index
  - Prove $\lim_{n\rarr\infty}a_n= l$ 无法之法，套用定义
    Discussion: $|a_n-l|<\varepsilon\Rarr\text{ solve for }N_\varepsilon.$
    Proof: $\forall \varepsilon>0,\text{ take }N_\varepsilon=...,\text{ we have... }\Rarr |a_n-l|<\varepsilon.\blacksquare$
    Example 1: Prove $\lim_{n\rarr\infty}\sqrt{1-\frac1n}=1$ .
    Hw1 T1: Back to index
  (i) $\Larr$ false. Take $\varepsilon=\frac12|a_N-l|$ .
  (ii) $\Larr$ take $m=\lceil\frac1\varepsilon\rceil$ , $\Rarr$ take $\varepsilon=\frac1m$ .
  (iii) $\Larr$ take $\delta=\varepsilon$ . Let the largest index of the finitely many $a_n$ $\notin(l-\varepsilon,l-\varepsilon)$ be $N$ .
  $\Rarr$ let $\varepsilon'=\min(\varepsilon,\delta)$ .

$N$ 变脸法：公卿在哪 (take $N=...$ 的操作)
- Hw1 T4 奇偶子列分别收敛于同一值，证原序列也收敛到那个值 Back to index
  Take $N_3=\max(N_1,N_2)\Rarr N_4=2N_3+1.$
- Qz1 T3 奇偶子列都 $\rarr\infty$ ，证原序列也去天堂 Back to index
  $\forall \varepsilon>0,\exists N_1,s.t.~a_{2k}>\varepsilon,\forall k\ge N_1\\\forall \varepsilon>0,\exists N_2,s.t.~a_{2k+1}>\varepsilon,\forall k\ge N_2$
  Take $N_3=\max(2N_1+1,2N_2+1)\Rarr\forall \varepsilon>0,\exists N_3,s.t.~a_{n}>\varepsilon,\forall n\ge N_3$ .
- Hw3 T3.2 易错： $N$ 的取值只能依赖于 $\varepsilon$ , 不能依赖于 $p,n$ , 因为需要满足 $\forall p,n(\ge N)$ . Back to index
  Counterexample: $a_1=0,a_n=\sum_{k=1}^{n-1}\frac1k$ .

中流寻砥柱
- Prove: UF-2 $\Rarr$ UF-1.
  We only need to consider $\varepsilon\ge\varepsilon_0$ . $\forall \varepsilon\ge\varepsilon_0,\exists N(\frac{\varepsilon_0}2)~s.t.~|a_n-l|\le c\cdot\frac{\varepsilon_0}2<c\varepsilon_0\le c\varepsilon,\forall n\ge N(\frac{\varepsilon_0}2).\blacksquare$

Triangle Inequality
- Prove: $\{a_n\}\text{ is convergent}\Rarr \{a_n\}\text{ is Cauchy}$
  Let $l=\lim a_n.\Rarr\forall n,m\ge N,|a_n-a_m|=|a_n-l+l-a_m|\le|a_n-l|+|a_m-l|<2\varepsilon. \blacksquare$
- Hw1 T2: Prove: convergence of $\{a_n\}\Rarr$ convergence of $\{|a_n|\}$ . Back to index
  Key: $||a_{n}|-|a||\le|a_{n}-a|$ . Maybe not exactly triangle inequality but…
- Hw3 T3.1

Divide and Conquer
Common ways to divide:
1. Fundamental: 断点（the birthday hoax），点之前利用finite # of #取max，点之后利用极限的定义和triangle inequality.
  1. Proof of 1.1, Toeplitz Theorem,
  1. Hw1 T5.2,
1. 先写出需要证明的式子，化出尽可能多的已知式，如 $|a_n-a|$ ，然后再因地制宜地划分。
  1. Proof of 1.3.3, 1.3.4.
- Basic 断点
  - Hw1 T5.2: Back to index
    Alternatively,
    $\frac{a_1+...+a_n}{n}\geq \frac{a_1+...+a_{N-1}}{n}+\frac{n-N+1}{n} M$
    Take $\liminf$ on both sides,
    $\liminf \frac{a_1+...+a_n}{n}\geq M$
    Therefore, $\forall M,$ $M<M+1\leq \liminf \frac{a_1+...+a_n}{n}$ , hence there exists N such that $\frac{a_1+...+a_n}{n}>M$ , hence $\lim \frac{a_1+...+a_n}{n}=\infty$ .
- Telescoping: $x_n=x_0+(x_1-x_0)+(x_2-x_1)+...+(x_n-x_{n-1})$
  - Hw3 T4: (1) $\{a_n\}_{n=0}^{\infty}$ is convergent $\Leftrightarrow \sum_{n=1}^{\infty} (a_n-a_{n-1})$ is convergent. (2) In particular, if $\sum_{n=1}^{\infty} |a_n-a_{n-1}|$ is convergent, then $\{a_n\}$ is convergent. $\{a_n\}_{n=0}^{\infty}$ is convergent $\Leftrightarrow \sum_{n=1}^{\infty} (a_n-a_{n-1})\text{ is convergent}\overset{3.3}{\Larr}$ $\sum_{n=1}^{\infty} |a_n-a_{n-1}|$ is convergent. Back to index
    (1) 抓紧 $s_n=a_n-a_0.$
    “ $\Larr$ ”
    Let $s_n=\sum_{k=1}^na_k-a_{k-1}=a_n-a_0.$ $\sum_{n=1}^{\infty} (a_n-a_{n-1})$ is convergent $\Rarr\lim s_n$ exists as a finite number. $a_n=s_n+a_0\Rarr\lim s_n+a_0=\lim a_n\Rarr\{a_n\}_{n=1}^\infty$ is convergent.
    “ $\Rarr$ ”
    $\{a_n\}_{n=1}^\infty$ is convergent $\Rarr\lim a_n$ exists as a finite number $\Rarr\lim(a_n-a_0)=\lim a_n-a_0=\lim s_n\Rarr\sum_{n=1}^{\infty} (a_n-a_{n-1})$ converges.
    (2)
    $\sum_{n=1}^\infty|a_n-a_{n-1}|$ converges $\overset{3.2}{\Rarr}\forall\varepsilon>0,\exists N~s.t.~\sum_{k=m+1}^n|a_k-a_{k-1}|<\varepsilon,\forall m+1>N\Rarr|a_n-a_m|=|\sum_{k=m+1}^n(a_k-a_{k-1})|\le \sum_{k=m+1}^n|(a_k-a_{k-1})|<\varepsilon,\forall m+1>N\Rarr\{a_n\}$ converges.
  - Hw3 T3 (iii): $\{a_n\}$ satisfies $|a_{n+p}-a_n|\leq \frac{p}{n^2}, \forall$ integer $n,p\geq1$ , prove that $\{a_n\}$ is Cauchy. Back to index
    WLOG, let $p=1.$ Notice that
    $|a_n-a_m|=|a_n-a_{n-1}+a_{n-1}-a_{n-2}+...+a_{m+1}-a_{m}|$
    $\le |a_n-a_{n-1}|+...+|a_{m+1}-a_m|$
    $\leq \frac{1}{(n-1)^2}+...+\frac{1}{m^2}$
    But $\sum \frac{1}{x^2}$ is convergent, hence
    $\forall \varepsilon>0, \exist N, s.t.\, |\sum_{k=m}^{n-1} \frac{1}{k^2}|=\frac{1}{(n-1)^2}+...+\frac{1}{m^2}<\varepsilon, \forall n \geq m \geq N$ , as desired.
Common ways to conquer:
1. Convergence $\Rarr$ boundedness, always $\le M$
1. $N_\varepsilon$ 之前，finite, always $\le\max\{a_1,...a_{N_\varepsilon}\}$ ; $N_\varepsilon$ 之后，always $\le l+\varepsilon$
- 你们一起上吧，我能打十个 (放缩求和的时候，多让它几项也无妨)
  - Rearrangement 1: If $\sum a_n$ converges absolutely, then any rearrangement $\sum a_{k_n}$ also converges absolutely. Moreover, $\sum a_{k_n}=\sum a_n$
    Let $\sum a_n'$ be a rearrangement, with partial sums $s'_n$ . Given $\forall \varepsilon>0$ , there exists an integer $N$ such that $n \geq N$ implies $\sum_{k=N}^n |a_k|<\varepsilon$
    Now choose $p$ such that the integer $1,2,...,N$ are all contained in the set $k_1,...,k_p$ . Hence if $n>p$ , the numbers $a_1,...,a_N$ will cancel in the difference $|s_n-s_n'|$ . After the elimination, all the elements left in the partial sum have a index greater than $N$ . Let $ss_n$ and $ss_n'$ be the partial sums after eliminating common elements, then
    $|s_n-s'_n|=|ss_n-ss'_n|\leq |ss_n|+|ss'_n|\leq \sum_{k=N}^{\infty} |a_k|$
    But $\sum_{k=N}^{\infty} |a_k|=\lim_{n \to \infty} \sum_{k=N}^{n} |a_k|\leq \varepsilon$ , hence we have
    $|s_n-s'_n|\leq\varepsilon, \forall n>p$
    Hence $\lim (s_n-s'_n)=0$ , $\lim s_n'=\lim s_n$
  - Rearrangement 2 (Hw4 T4): Back to index
    Let $\sum a_n$ be a convergent series. Suppose $\sum a'_n$ is a rearrangement of $\sum a_n$ satisfying that exist fixed integer $m\geq1$ such that $\forall n\geq 1$ , $a'_n=a_{n'}$ , where $|n-n'|\leq m$ . Prove that $\sum a_n'$ converges and equals $\sum a_n$ .
    Notice that in this case:
    $\{a'_k\}_{k=1}^n \subset \{a_1,...,a_{n-m},a_{n-m+1},...,a_n,...,a_{n+m}\}$
    $\{a_k\}_{k=1}^n=\{a_1,...,a_n\}$
    The first $n-m$ terms in the original sequence must cancel out in $s_n'-s_n$ as for any $k'\in[1,n-m]$ we can find $k \in[1,n]$ such that $a'_k=a_{k'}$ .
    After elimination, all the elements remaining in the partial sums have a index greater than $n-m$ and less or equal to $n+m$ . Denote $ss_n$ and $ss'_n$ be the partial sums after elimination, then
    $|s_n'-s_n|=|ss'_n-ss_n|\leq|ss_n'|+|ss_n|\leq \sum_{k=n-m+1}^{n+m} |a_k|$
    Since $\sum a_n$ is convergent, by divergence test, we must have $\lim a_n=0$ .
    Hence as long as $n$ is bigger enough, then $n-m-1$ is bigger enough (because $m$ is a fixed constant), then $\forall k\in[n-m+1,n+m]$ , $|a_k|<\varepsilon$ .
    Therefore, $\forall \varepsilon>0, \exists$ $N$ , $s.t., |s_n'-s_n|\leq 2m\varepsilon, \forall n\geq N$ , thus $\lim (s'_n-s_n)=0$ , $\lim s'_n=\lim s_n$
- Prove quotient rule: $\lim_{n \to \infty} \frac{a_n}{b_n}=\frac{\lim_{n\to \infty} a_n}{\lim_{n\to \infty}b_n}$
  $|\frac{a_n}{b_n}-\frac{a}{b}|=|\frac{a_nb-ab_n}{b_nb}|=|\frac{a_nb-a_nb_n+a_nb_n-ab_n}{b_nb}|\leq \frac{|a_n||b_n-b|}{|b_nb|}+\frac{|b_n||a_n-a|}{|b_nb|}$
  Since convergent sequence is bounded, we can find a $M$ , such that $|a_n|,|b_n|\leq M$ .
  $\Rarr|\frac{a_n}{b_n}-\frac{a}{b}|\leq \frac{M(|b_n-b|+|a_n-a|)}{|b_nb|}$
  Since $\lim a_n=a, \lim b_n=b$ , then
  $\forall \epsilon>0, \exist N(\epsilon), s.t. |a_n-a|<\epsilon, |b_n-b|\leq \epsilon,\forall n\geq N(\epsilon)$ , thus we have
  $\Rarr|\frac{a_n}{b_n}-\frac{a}{b}|\leq \frac{2M\epsilon}{|b_nb|}$
  We then want to show that $\frac{2M}{|b_nb|}$ is smaller than a constant, which is equivalent to showing $|b_nb|$ is bigger than a constant. How do we do this. Observe that if $n$ is large enough, we have $|b|-|b_n|\le|b_n-b|<\epsilon\Rarr|b_n|\ge|b|-\epsilon$ now all we need to do is make $\epsilon$ a constant, simple, we just take $\epsilon=\frac{|b|}{2}\Rarr|b_nb|\ge\frac{|b|^2}{2},\forall n\ge N_{\frac{|b|}{2}}$
  $\Rarr |\frac{a_n}{b_n}-\frac{a}{b}|<\frac{2M}{\frac{|b|^2}{2}}\cdot \epsilon,\forall n\ge\max\{N(\epsilon),N_{\frac{|b|}{2}}\}$
- Example 5: $\lim_{n \to \infty} a_n= A, \lim_{n \to \infty} b_n=B, c_n=\frac{a_1b_n+...+a_nb_1}{n}$ . Show that
  $|\frac{a_1b_n+...a_nb_1}{n}-AB|=|\frac{(a_1b_n-AB)+...+(a_nb_1-AB)}{n}|$
  $\leq |\frac{a_1b_n-AB}{n}|+...+|\frac{a_nb_1-AB}{n}|$
  Observe $|a_1b_n-AB|=|a_1b_n-Ab_n+Ab_n-AB|\leq b_n|a_1-A|+A|b_n-B|$
  Similarly, $|a_k-b_{n-k+1}-AB|=b_{n-k+1}|a_k-A|+A|b_{n-k+1}-B|$
  Therefore,
  $|\frac{a_1b_n-AB}{n}|+...+\frac{a_nb_1-AB}{n} | \leq \frac{|b_n||a_1-A|}{n}+\frac{|A||b_n-B|}{n}+...+\frac{|b_1||a_n-A|}{n}+\frac{|A||b_1-B|}{n}$
  $\leq |A|\frac{|b_1-B|+...+|b_n-B|}{n} +\frac{|b_n||a_1-A|+...+|b_1||a_n-A|}{n}$
  We already know that $\frac{|b_1-B|+...+|b_n-B|}{n} < 2\epsilon, \frac{|a_1-A|+...+|a_n-A|}{n}<2\epsilon$ , the first term is nice, but the second term is awful, we want to have a common factor.
  Notice that $\{b_n\}$ converges implies that $\{b_n\}$ is bounded, hence $\exist M, s.t. \,\,b_n \leq M, \forall n$
  Consequently,
  $|A|\frac{|b_1-B|+...+|b_n-B|}{n} +\frac{|b_n||a_1-A|+...+|b_1||a_n-A|}{n}$
  $\leq |A|\frac{|b_1-B|+...+|b_n-B|}{n} +M\frac{|a_1-A|+...+|a_n-A|}{n} \leq 2|A|\epsilon+2M\epsilon$
- Hw1 T5.3 (Toeplitz Theorem): Suppose $\forall$ integer $n,k\geq 1,t_{n_k}\geq 0$ . $\sum_{k=1}^{n} t_{n_k}=1$ and $\lim_{n\to\infty} t_{n_k}=0$ . Prove that if $\lim_{n\to \infty} a_n=l$ , $l$ is a finite number, then $\lim_{n\to \infty} \sum_{k=1}^n t_{n_k}a_k=l$ . Back to index
  Proof of Fact 1.1 的“虚张声势”版，思路非常基础：断点然后分而治之。
  $|\sum_{k=1}^n t_{n_k}a_k-l|\overset{\sum_{k=1}^{n} t_{n_k}=1}{=}|\sum_{k=1}^nt_{n_k}(a_k-l)| \leq \sum_{k=1}^n t_{n_k}|a_k-l|$
  Since $\lim a_n=l$ , then $\forall \epsilon>0, \exist N(\epsilon), s.t. \,\, |a_n-l|<\epsilon, \forall n \geq N$ .
  Therefore,
  $\sum_{k=1}^{n} t_{n_k}|a_k-l|\leq \sum_{k=1}^{N(\epsilon)-1} t_{n_k}|a_k-l|+\epsilon \sum_{k=N(\epsilon)}^{n} t_{n_k}$
  Exploit the fact that $|a_k-l|\leq M+|l|$ and $\sum_{k=N(\epsilon)}^n t_{n_k} \leq \sum_{k=1}^n t_{n_k}=1$ , we have
  $\sum_{k=1}^{N(\epsilon)-1} t_{n_k}|a_k-l|+\epsilon \sum_{k=N(\epsilon)}^{n} t_{n_k} \leq (M+|l|) \sum_{k=1}^{N(\epsilon)-1} t_{n_k}+\epsilon$
  We want the first term to be less than $c \epsilon$ . Using the fact that $\lim_{n \to \infty} t_{n_k}=0$ ,
  Take $\epsilon'=\frac{1}{(M+|l|)(N(\epsilon)-1)}\epsilon$ , then $\exist N'(\epsilon')$ , $s.t.\,\,\, t_{n_k} \leq \frac{1}{(M+|l|)(N(\epsilon)-1)}\epsilon, \forall n \geq N',\forall k$
  Then $(M+|l|) \sum_{k=1}^{N(\epsilon)-1} t_{n_k}+\epsilon \leq 2\epsilon. \blacksquare$
  Special case given as Example 2 in W1L2: Suppose $\lim a_n=l,\text{ prove that }\lim \frac{a_1+...+a_n}{n}=l.$
  Hw1 T5.4: Back to index
  key: let $t_{nk}=\frac{C^k_n}{2^n-1}$ . Then try to prove 1) $\sum_{k=1}^nt_{nk}=1$ , 2) $\lim_{n\rarr\infty} t_{nk}=0$ .
  (最后再乘一个 $\frac{2^n-1}{2^n}$ , 而 $\lim \frac{2^n-1}{2^n}=1$ )
  证明 $\lim_{n\to \infty} \frac{C_n^k}{2^n-1}=0$ ,
  $\lim \frac{C_n^k}{2^n-1}=\lim\frac{\frac{n\times(n-1)\times...\times (n-k+1)}{k!}}{2^{n}-1}=\lim\frac{n^k}{(2^n-1)k!}\frac{n\times (n-1)\times... \times (n-k+1)}{n^k}$
  $=\lim \frac{n^k}{(2^n-1)k!} \frac{n}{n}\times \frac{n-1}{n}\times...\times \frac{n-k+1}{n}$
  $=\lim\frac{n^k}{(2^n-1)k!}=0$
  (fixed k, let n go to infinity. 指数比阶乘大)
- Hw3 T12: Add your own flavor Back to index (它在这)
  - Suppose $\forall n \geq 1, a_n \geq 0$ , and $\sum_{n=1}^{\infty} a_n$ converges with $a_{n+1}\leq a_n$ . Prove that $\lim_{n \to \infty} (na_n)=0$ .
    Solution family 1
    Notice that $0\leq \frac{n}{2} a_n \leq \sum_{k=\lceil\frac n2\rceil}^{n} a_k \leq \sum_{k=\frac{n}{2}}^{\infty} a_k$
    $\sum_{k=\lceil\frac n2\rceil}^{\infty} a_k= \sum_{k=1}^{\infty} a_k- \sum_{k=1}^{\lceil\frac n2\rceil-1}a_k$ is the remainder of $\sum_{n=1}^{\infty} a_n$
    Hence $\lim_{n\to \infty} \sum_{k=[\frac{n}{2}]}^{\infty}a_k=\sum_{k=1}^{\infty} a_k-\sum_{k=1}^{\infty} a_k=0$ (F3.2 尾巴趋于0, or F3.0殊始同归）
    By squeezing principle, $\lim \frac{n}{2} a_n=0$ , hence $\lim na_n=0$ .
    Solution family 2 (Cind’s favorito!)
    为了让 $na_n$ 出现在 $<\varepsilon$ 左边，我们需要构造一串和使得他们的角标能让项的个数与 $n$ 有关。又由于 $a_n\darr$ , 所以需要构造 $x_?+...+x_n$ 单减所以往前跑，若单增，要放大，就往后跑。
    $\forall \varepsilon>0,\exists N,\text{ s.t., WLOG, }\forall m>n>N,a_{n+1}+...+a_m\overset{\text{fact 3.2}}{<}\varepsilon.$ Let $N'=2N+3$ , then $\lfloor\frac n2\rfloor>N+1,\forall n\ge N'\Rarr \frac n2 a_n<a_{\lfloor\frac n2\rfloor}+...+a_n<\varepsilon.$ Therefore $\{na_n\}$ converges to 0.
- Hw4 T5 plus (Cesaro-convergence): If $\{s_n\}$ is a complex sequence, define its arithmetic means $\sigma_n$ by $\sigma_n=\frac{s_1+...+s_n}{n}$ . Back to index
  - (i) Construct a sequence $\{s_n\}$ which does not converge, although $\lim \sigma_n=0$ .
    Let $s_n=(-1)^n$ . Hence $\sigma_n=0$ is $n$ is even and $\frac{1}{n+1}$ if $n$ is even. Thus $\sigma_n\to 0$ though $s_n$ has no limit.
  - (ii) If $\lim s_n=s$ , prove that $\lim \sigma_n=s$ . (Usual convergence implies Cesaro-convergence)
    $|\sigma_n-s|=|\frac{(s_1-s)+...+(s_n-s)}{n}| \leq \frac{1}{n}[|s_1-s|+...+|s_n-s|]$
    Since $\lim s_n=s$ , then $\forall \varepsilon>0, \exist N(\epsilon), s.t.|s_n-s|<\varepsilon, \forall n\geq N(\varepsilon)$
    Hence $|\sigma_n-s_n|\leq \frac{1}{n}[|s_1-s|+...+|s_{N(\varepsilon)-1}-s|+(n-N(\epsilon)+1)\varepsilon]$
    Moreover, given $\{s_n\}$ is convergent, $\{s_n\}$ is bounded, hence $|s_1-s|\leq M+|s|$ .
    Therefore, $|\sigma_n-s_n|\leq \frac{(N(\varepsilon)-1)(M+|s|)}{n}+\varepsilon<2\varepsilon$ if $n \geq \frac{N(\varepsilon-1)(M+|s|)}{\varepsilon}$ .
    Essentially the same with Example 2 and Hw1 T5 Toeplitz Theorem.
  - Can it happen that $s_n>0$ for all $n$ and that $\limsup s_n=\infty$ , although $\lim \sigma_n=0$ .
    Let $s_n=\frac{1}{n}$ if $n$ is not a perfect cube and $s_n=\sqrt[3]{n}$ if $n$ is a perfect cube (Construct a bad guy). Then if $k^3\leq n \leq (k+1)^3$ we have
    $\sigma_n \leq \frac{1}{n}\sum_{m=1}^n \frac{1}{m}+\frac{1}{n}\sum_{j=1}^k j= \frac{\sum_{m=1}^n \frac{1}{m}}{n}+\frac{1}{n} \frac{k(k+1)}{2}$ (为了形式好看多让了几项 $\frac{1}{n}$ )
    The first sum on the right tends to zero by part (a) applied to the sequence $s_n=\frac{1}{n}$ . As for the last term, since $n\geq k^3,$ we have $\frac{k^2+k}{2n}\leq \frac{k^2}{2k^3}+\frac{k}{2k^3}=\frac{1}{2k}+\frac{1}{2k^2}$ , which tends to zero as $n$ tends to zero as $k\to \infty$ . Since $(k+1)^3>n$ , it follows that $k$ tends to infinity as $n$ tends to infinity, and hence we have $\sigma_n\to 0$ , even though $s_{n^3} \to \infty$ (hence $\overline{\lim} s_n=\infty$ ).
  - (iii) Put $a_n=s_n-s_{n-1}$ . Show that $s_n-\sigma_n=\frac{1}{n}\sum_{k=2}^n (k-1)a_k$ . Assume that $\lim (na_n)=0$ and that $\{\sigma_n\}$ converges. Prove that $\{s_n\}$ converges.
    $s_n-\sigma_n=\frac{n(a_1+...a_n)-s_1...-s_n}{n}=\frac{a_2+...+(n-1)a_n}{n}=\frac{1}{n}\sum (k-1)a_k$
    If $\lim na_n=0$ , then $\forall \varepsilon>0, \exist N,s.t. |a_n|\leq |na_n|\leq\varepsilon,\forall n\geq N$ , hence $\lim a_n=0$ , and $\lim (n-1)a_n=0$ . Then apply part (a) to the sequence $(n-1)a_n$
    we have $\lim (s_n-\sigma_n)=\lim \frac{\sum_{k=1}^n (k-1)a_k}{n}=\lim (n-1)a_n=0$ , hence $\lim s_n=\lim \sigma_n$
  - (iv)

Powerful $\varlimsup,\varliminf$ 一言不合就上下极限
通过一定存在的东西（ $\varlimsup,\varliminf$ ）寻找不确定存在的东西（ $\lim$ ）
1. F2.5 (Useful trick) $\lim a_n=l\Lrarr\varlimsup_{n\rarr\infty} |a_n-l|=0.$
- Fix an arbitrary $m$ in a set, then do limit/sup/inf
  - Hw2 T5: Suppose $a_n\geq0,\forall n\geq1$ . Prove that $\overline{\lim}_{n\to \infty} a_n^{\frac{1}{n}}\leq 1$ if and only if $\lim \frac{a_n}{l^n}=0$ for any $l>1$ .Back to index
    $\Rightarrow$
    For any $l>1$ , take $\varepsilon=\frac{l-1}{2}$ , then $\overline{\lim} a_n^{\frac{1}{n}}<1+\varepsilon<l$ ,
    By fact 2.3.1, there exists $N$ , s.t. $a_n^{\frac{1}{n}}<(1+\epsilon)^n$ as long as $n\geq N$ .
    which implies $a_n<(1+\varepsilon)^n, \forall n \geq N$ .
    hence $0\leq \frac{a_n}{l}\leq (\frac{1+\varepsilon}{l})^n$ . By squeezing principle, $\lim \frac{a_n}{l}=0$ .
    Alternatively, start from $\varlimsup (a_n)^\frac1n\le 1\le \frac{l+1}2$ .
    $\Leftarrow$
    For any $l>1$ , take $\varepsilon=1$ , then there exists $N$ such that $\frac{a_n}{l^n}<1,$ $\forall n\geq N$
    Hence $a_n^{\frac{1}{n}}<l$ , $\forall n\geq N$ .
    Take limsup on both sides, we have $\overline{\lim} a_n^{\frac{1}{n}}\leq l$
    Here comes our ole friend again: since $\overline{\lim}a_n^{\frac{1}{n}}\leq l$ for all $l\in (1,\infty)$ , we have
    $\overline{\lim} a_n^{\frac{1}{n}}\leq \inf{(1,\infty)}=1$
    @The Grand Inquisitor I like this $\char"263b$ . Ole friend: proof of theorem “root test more general than ratio test.”
  - Hw3 T7: m 进制表示法 Back to index
    Suppose $\forall n \geq 1, x_n \geq 0$ and $x_{m+n}\leq x_n+x_m. \forall n,m\geq1$ . Prove that $\lim \frac{x_n}{n}$ exists (as a finite number).
    Hint: Fix an arbitrary integer $m \geq 1,$ then $\forall n \geq 1$ , $\exist$ integer $p$ and $r$ s.t. $n=pm+r$ . Try to show $\limsup \frac{x_n}{n} \leq \inf \{\frac{x_m}{m}\}_{m=1}^{\infty}$ .
    Solution
    Fix $m$ , then $\forall n$ , $n=pm+r$ . Hence
    $x_n=x_{pm+r} \leq p x_{m}+x_r$ （这一步分出来挺关键的）
    $\Rightarrow \frac{x_n}{n} \leq \frac{px_m}{pm+r} +\frac{x_r}{n}$
    Take limsup on both sides, (since we don’t know whether $\lim \frac{x_n}{n}$ exists, in these cases we often take lim sup/inf.)
    $\limsup_{n\to \infty}= \limsup_{n \to \infty} \frac{x_m}{m+\frac{r}{p}}+\frac{x_r}{n}$ (having an arbitrary something on the numerator and a heaven-going something on the demoninator comes really handy.)
    Since $r<m$ , $x_r$ is bounded by $\max\{x_1,...,x_{m-1}\}$ . Hence both $\frac{r}{p}$ and $\frac{x_r}{n}$ converges to zero as $n$ goes to infinity, which implies that the RHS actually has a limit. $\limsup \frac{x_m}{m+\frac{r}{p}}+\frac{x_r}{n} =\lim (\frac{x_m}{m+\frac{r}{p}}+\frac{x_r}{n})=\frac{x_m}m{}$ . Alternatively, since $\frac{r}{p}>0,\frac{x_m}{m+\frac{r}{p}}+\frac{x_r}{n}<\frac{x_m}{m}+\frac{x_r}{n}$ .
    Therefore, we conclude that $\limsup _{n\to \infty} \frac{x_n}{n}\leq \frac{x_m}{m},\forall m$ .
    Hence $\limsup_{n\to \infty} \frac{x_n}{n} \leq \inf\{\frac{x_m}{m}\}_{m=1}^{\infty}$ , since m is arbitrary.
    Thus $\liminf_{n \to \infty} \frac{x_n}{n} \overset{\triangle}{\geq} \inf\{ \frac{x_n}{n}\}_{n=1}^{\infty} \geq \limsup _{n\to \infty} \frac{x_n}{n}$ (the form of $\varliminf\ge\varlimsup$ is often what we desire when we use these notations in the first place, 夹一个 $\lim$ 出来！), which implies $\limsup \frac{x_n}{n}=\liminf\frac{x_n}{n}$ and $\lim \frac{x_n}{n}$ exists. $\triangle$ is due to $y_n=\inf\{a_n,a_{n+1},...\}$ monotonically increases.

系数堆积法（ $\alpha^p,\alpha\bigcirc 1$ ）
一般是利用/构造等比数列， $\alpha<1$ 时用于证convergence, $>1$ 时用于证divergence.
- Hw3 T5: Fixed point Back to index
  - Suppose $f$ is differentiable on $(-\infty, +\infty)$ , satisfying $|f'(x)|\leq \alpha, \forall x \in (-\infty, \infty)$ , where $\alpha$ is a constant belonging to $(0,1)$ . Fix an arbitrary $x_0\in(-\infty, +\infty)$ . Define sequence $\{x_n\}_{n=1}^{\infty}$ as: $x_1=f(x_0), x_2=f(x_1),...,x_{n+1}=f(x_n)$ . Prove that $\lim_{n \to \infty} x_n$ exists as a finite number $x_{\infty}$ , with $x_{\infty}=f(x_{\infty})$ . [Hint: Mean Value Theorem].
  - Solution
    Notice that $x_n=x_0+\sum_{k=1}^{n} (x_k-x_{k-1})$
    To prove $\lim x_n$ exists, it is sufficient to prove $\sum_{k=1}^{\infty} |x_k-x_{k-1}|$ converges since absolute convergence implies convergence.
    Then we compute $|x_k-x_{k-1}|$ by induction:
    $|x_1-x_0|=|f(x_0)-x_0|$
    $|x_2-x_1|=|f(x_1)-f(x_0)|=|f'(\xi)(x_1-x_0)|\leq \alpha |x_1-x_0|$
    By induction
    $|x_n-x_{n-1}| \leq \alpha^{n-1} |x_1-x_0|$
    Hence $\sum_{n=1}^{\infty} |x_{n}-x_{n-1}|\leq \sum_{n=1}^{\infty} \alpha^{n-1} |x_1-x_0| =\frac{1}{1-\alpha} |x_1-x_0|$ , a finite number.
    By comparison test, $\sum_{n=1}^{\infty} |x_n-x_{n-1}|$ converges.
    Hence $\lim x_n= x_0+ \sum_{n=1}^{\infty} |x_n-x_{n-1}|$ exists & finite. $\lim_{n\rarr\infty} f(x_n)=\lim_{n\rarr\infty} x_{n+1}=\lim_{n\rarr\infty} x_{n}=x_\infty=f(x_\infty).$
- Root Test: if $\alpha=\limsup |a_n|^{\frac{1}{n}}<1,$ then $\sum |a_n|$ converges
  Take $\beta\in (\alpha,1)$ , then by fact 2.3.1, there exists $N$ such that $|a_n|^{\frac{1}{n}}<\beta$ as long as $n \geq N$ . Hence $|a_n|^{n}<\beta^n, \forall n \geq N$ . Since $\sum \beta^n$ as a geometric series converges, by comparison test, $\sum |a_n|$ converges.
- Ratio Test: if $\limsup |\frac{a_{n+1}}{a_n}|<1$ , then $\sum |a_n|$ converges
  Take $\beta\in (0,1)$ , by ole result, there exists $N$ such that $|\frac{a_{n+1}}{a_n}|<\beta$ as long as $n$ is larger than $N$ . Hence
  $|a_{N+1}|<\beta |a_N|$
  $|a_{N+2}|<\beta |a_{N+1}|<\beta^2 |a_N|$
  By induction: $|a_{N+p}|<\beta^p |a_N|$
  Since $\sum_{n=N}^{\infty} \beta^n |a_N|$ converges, by comparision test, $\sum_{n=N}^{\infty} |a_n|$ converges, hence $\sum |a_n|$ converges.
- Prove that Root Test is more general than Ratio Test
  - $\limsup |a_n|^{\frac{1}{n}}\leq \limsup |\frac{a_{n+1}}{a_n}|$
  - Proof
    Denote $\alpha=\limsup |a_n|^{\frac{1}{n}}$ and $\beta=\limsup |\frac{a_{n+1}}{a_n}|$
    Take $r\in (\beta, \infty)$ , by ole result, there exists $N$ such that $|\frac{a_{n+1}}{a_n}|<r$ as long as n is larger than $N$ . Therefore,
    $|a_{N+1}|<r |a_N|$
    $|a_{N+2}|<r^2|a_N|$
    By induction, we have $|a_n|< r^{n-N}|a_N|, \forall n\geq N$
    Hence $|a_n|^{\frac{1}{n}}<r^{\frac{n-N}{n}}|a_N|^{\frac{1}{n}}$
    Take limsup on both sides (RHS actually has limit) we have
    $\limsup |a_n|^{\frac{1}{n}}< \limsup r^{\frac{n-N}{n}}|a_N|^{\frac{1}{n}}=r$
    Since $\forall r \in (\beta,\infty)$ , $\alpha=\limsup|a_n|^{\frac{1}{n}}<r$ , we have $\alpha\leq \inf(\beta,\infty)=\beta$

The Bad Guy(s) - Proof by constructing/finding a special subsequence (Tian’s Favorite Trick)
抓住 $\forall$ 之后的东西：它是自由的。在一般的情况下是标准定义里的 $\varepsilon$ , 如果任务有变，则看变之后 $\forall$ 之后是啥。然后I have a dream分析之后推出需要take it to be啥，然后大功告成。
- Formulating the rigorous definition of $\lim_{n\to \infty} a_n \neq l$ from naive version
  The naive version is derived from taking the opposite of the definition for $\lim_{n\to \infty} a_n =l$
  $\exist \epsilon_0,s.t.\,\, \forall N, \exist n(N)\geq N, s.t. |a_{n(N)}-l| \geq \epsilon_0$
  The refined definition using the language of subsequence:
  $\exist \epsilon_0, s.t.\,\, \exist n_1<n_2<...<n_k<...,s.t.\,\, |a_{n_k}-l| \geq \epsilon_0, \forall k$
  We now show the construction process of $n _1,n_2,..,n_k,...$
  (Remark: the key is that these indexes must be increasing)
  Take $N_1=1$ , then there exists $n(N_1)\geq N_1$ such that $|a_{n(N_1)}-l| \geq \epsilon_0$ .
  Relabel $n(N_1)$ as $n_1$ .
  To guarantee $n_2 > n_1$ , we take $N_2=n_1+1$ , then there exist $n(N_2) \geq N_2$ such that $|a_{n(N_2)}-l| \geq \epsilon_0$ . Relabel $n(N_2)$ as $n_2$ and we know $n_2 \geq N_2 >n_1$ .
  Continue this process, we can have a sequence of bad guys.
- Prove Fact 1.6.
  - Fact 6: Suppose $\{a_n\}$ satisfies: $\forall$ subsequence of $\{ a_n\}$ , $\exist$ sub-subsequence $\{a_{n_k}\}_{k=1}^{\infty}$ , s.t. $\lim_{k\to \infty} a_{n_k}$ exists & $=l$ , where $l$ is independent of the choice of subsequence. Then $\lim_{k\to \infty} a_{n}$ exists & $=l$ .
  - Proof by contradiction
    Suppose otherwise ( $\lim_{n \to \infty} a_n \neq l$ ), then by the rigorous definition of $\lim_{n \to \infty} a_n \neq l$ , there exists a bad $\epsilon_0$ and a corresponding bad subsequence $\{ a_{n_k}\}$ such that $|a_{n_k}-l| \geq \epsilon_0, \forall k ~(\star)$
    Then for this subsequence, we cannot find a sub-sub sequence whose limit is $l$ . And how do we prove we cannot find such a subsubseq? Proof by contradiction again:
    Under the assumption of the problem, $\exists$ a subsubseq $\{a_{n_{k_m}}\}~s.t.~\lim a_{n_{k_m}}=l~(\square).\because \{a_{n_{k_m}}\}_{m=1}^\infty\sub\{a_{n_{k}}\}_{k=1}^\infty\overset{\star}{\Rarr}|a_{n_{k_m}}-l|\ge\varepsilon_0,\forall m\ge 1\Rarr \lim a_{n_{k_m}}\ne l$ , contradicting $\square$ .
- Example 6 MIDTERM - Would you like to be amazed? Back to index
  Let $\{ a_n\}$ be a bounded sequence satisfying $\lim (a_{2n}+ca_n)=0$ , where $c$ is a constant such that $|c| \geq 1$ . Show that $\lim a_n=0$ .
  - Orthodox: A jumble of facts, extremely high illustrative value!
    $\{a_n\}$ is bounded $\Rarr\exists R>0~s.t.~-R\le a_n\le R,\forall n\ge 1\Rarr -R\le \varliminf a_n\le \varlimsup a_n \le R$
    @The Grand Inquisitor 小天，这个最后一个 $\Rarr$ 是用的哪个fact啊，I buy it but I don’t feel extremely certain…
    @Cindy Zeng $R\geq \sup \{a_n\} \geq \overline{\lim} a_n, -R\leq \inf \{a_n\} \leq \underline{\lim} a_n$ @The Grand Inquisitor ok….
    Let $\varliminf a_n=m, \varlimsup a_n=M.$
    WTS $m=M=0$ , and use Fact 2 to prove $\lim a_n=0$ .
    $\lim (a_{2n}+ca_n)=0\overset{F2}{\Rarr}\varliminf(a_{2n}+ca_n)=\varlimsup(a_{2n}+ca_n)=0\\\overset{F6}{\Rarr}\varlimsup(a_{2n})+\varlimsup (ca_n)\ge \varlimsup(a_{2n}+ca_n)=0~(\bigstar)\\\overset{F3.2}{\Rarr}\exists$ subseq of $\{a_{2n}\}$ that converges to $\varlimsup(a_{2n})$ . Let this subseq be $\{a_{2n_m}\}$ , then $\{a_{2n_m}\}$ is also a subseq of $\{a_n\}\Rarr\varlimsup(a_{2n})\in E$ .
    $\overset{F3.3}{\Rarr}M=\sup E\ge\varlimsup(a_{2n})~(\clubs)$ , similarly, $m=\inf E\le\varliminf(a_{2n})~(\spades)$ $\char"263a$
    $\overset{\bigstar}{\Rarr}M+cM\ge 0\Rarr (1+c)M\ge 0.$ Similarly, $(1+c)m\le 0$ .
    Observe $ca_n=(a_{2n}+ca_n)-a_{2n}$
    Taking $\varlimsup$ on both sides we have $cM\overset{F7}{=}\lim(a_{2n}+ca_n)+\varlimsup(-a_{2n})\overset{F8}{=}-\varliminf(a_{2n})\overset{\spades}{\le}-m\Rarr cM+m\le0.$
    Taking $\varliminf$ on both sides we have $cm+M\ge 0.$
    If $c\ge 1$
    $\Rarr\begin{cases}cM+m\le 0\\cm+M\ge0\\m\le 0\\M\ge 0\end{cases}\Rarr\begin{cases}c^2M+cm\le 0\\cm+M\ge 0\end{cases}\Rarr(c^2-1)M\le 0\Rarr M\le 0\Rarr M=0.$ Similarly $m=0$ .
    If $c\le -1$
    $\Rarr\begin{cases}cM+m\le 0\\cm+M\ge0\\m\ge 0\\M\le 0(\Rarr cM\ge0)\end{cases}\Rarr m\le 0\Rarr m=0.$ Similarly $M=0$ . @The Grand Inquisitor @Cindy Zeng 哦哦，忘了前面条件是 $(1+c)M\geq0$ 了😭
    Therefore we have $\lim a_n=0$ .
    @The Grand Inquisitor呜呜小天，我觉得我把自己绕进去了。怎么从以下四个式子直接推M=m=0? 我知道如果分类讨论c<-1 还是c>1 怎么得出来，但是如果不分开，直接一起，我咋推不出来了呢？
    $\Rarr\begin{cases}cM+m\le 0\\cm+M\ge0\\(1+c)m\le 0\\(1+c)M\ge 0\end{cases}\Rarr$ ?
  @Cindy Zeng 我觉得合一起是不好推，因为涉及不等式两边同乘c是否要变号的问题，而它前面的式子又搞不出来 $|c|$ ，这应该是police后来推广的强化版，原版c>1的话就没问题。 @Cindy Zeng 等等 $c<-1$ 的话变号之后两个不等式都是 $\geq0$ , 那不能得出它们相减是 $\leq0$ 的啊
  @The Grand Inquisitor 换一个路径可以搞出来。我吃完饭回来写给你如果你没想出来的话。
  - Police: Proof by contradiction
    Suppose $\lim a_n \neq 0$ , then there exists a bad $\epsilon_0$ and a corresponding bad subsequence $\{a_{n_k}\}_{k=1}^{\infty}$ such that $|a_{n_k}|\geq \epsilon_0$ for big enough $n_k$ .
    On the other hand, by $\lim (a_{2n}+ca_n)=0$ , we have
    $\forall \epsilon, \exist N,s.t. |a_{2n}+ca_n| < \epsilon, \forall n \geq N$ , which implies
    $|c||a_n|-|a_{2n}|\leq|a_{2n}+ca_n|<\epsilon$ $\Rightarrow |a_{2n}|\geq |c||a_n|-\epsilon$
    Secret Room: I have a dream
    Since $|c|>1$ and we can find a $n_k$ such that $|a_{n_k}|>\epsilon_0$ , with an appropriate choice of $\epsilon$ , we hope to construct an exploding subsequence in the form of $a_{n_k} \geq \alpha^k \epsilon_0$ .
    To set off an explosion, we can choose $\epsilon=x \epsilon_0$ where $x$ satisfies $|c|-x >1$ . A customary choice is $x=\frac{|c|-1}{2}$ , then $|c|-x=\frac{|c|+1}{2} >1$ .
    Take $\epsilon=\frac{|c|-1}{2}\epsilon_0$ and take $n=max(N(\epsilon),n_1)$ , then we have
    $|a_{2n}| > |c| |a_n|-\epsilon>|c|\epsilon_0-\frac{|c|-1}{2}\epsilon_0 =\frac{|c|+1}{2}\epsilon_0$
    We relabel $m_1=2n$ and we take $m _2=2m_1$ , then we have
    $|a_{m_2}|>|c||a_{m1}|-\epsilon > |c| \frac{|c|+1}{2}\epsilon_0-\frac{|c|-1}{2}\epsilon_0= \frac{|c|^2+1}{2}\epsilon_0$
    Continue this process, we can construct a subsequence $\{a_{m_k}\}_{k=1}^{\infty}$ where
    $m_k=2^kn$ and $|a_{m_k}|>\frac{|c|^k+1}{2}\epsilon_0$
    But $\lim_{k \to \infty} \frac{|c|^k+1}{2} \epsilon_0=\infty$ , hence $\{a_{m_k}\}_{k=0}^{\infty}$ is unbounded, thus violates the condition that $\{a_n\}$ is bounded.
- Prove Fact 2.3.2: there always exists a subsequence whose limit is the limsup.
  思考怎么抽子列，能抽出一个limit是原序列limsup的子列？也就是n足够大的时候，subseq的元素 $\in (\bar l-\varepsilon,\bar l+\varepsilon)$ . 右边肯定满足，那怎么构建一个左边也满足的子序列？利用sup定义第二条就行。这个子序列不需要一开始就很接近，只要不断接近，最后无限接近，就好了。
  Since $\limsup a_n=\lim z_n=\bar{l}$ ,
  we have $\forall \epsilon, \exist N, s.t. \,\,\bar{l}-\epsilon<z_n<\bar{l}+\epsilon,\forall n \geq N$ .
  In the first round, we choose $\epsilon_1=\frac{1}{2}$ and take $n=N(\epsilon_1)$
  Since $z_n=\sup\{a_n,a_{n+1},...\}$ , by the definition of sup, there exists a $n_1 \geq n=N, s.t.$ $m \geq n=N, s.t.$
  $a_{n_1}> \bar{l}-\epsilon_1$
  Because $\bar{l}-\epsilon_1 <z_n$ . In addition, we also have $a_{n_1}\leq z_n<\bar{l}+\epsilon_1$ , so we $|a_{n_1}-\bar{l}|<\epsilon_1$
  In the second round, we choose $\epsilon_2=\frac{1}{4}$ , and let $N_2=max(N(\epsilon_2),n_1)$ .
  Similarly, we there exists a $n_2\geq N_2, s.t.$ $|a_{n_2}-\bar{l}| < \epsilon_2$ .
  In the third round, we choose $\epsilon_3=\frac{1}{8}$ , and let $N_3=max(N(\epsilon_3),n_2)$
  Then by the same logic, there exists a $n_3 \geq N_3,s.t. |a_{n_3} -\bar{l}|<\epsilon_3$ .
  Continue this process, we can construct a subsequence $\{a_{n_k}\}_{k=1}^{\infty},s.t.$
  $|a_{n_k}-\bar{l}| <\frac{1}{2^k}$
  Since $\lim_{k \to \infty} \frac{1}{2^k}=0$ and $\{\frac{1}{2^k}\}_{k=1}^{\infty}$ is monotone decreasing,
  we have $\forall \epsilon>0, \exist K, s.t.\,\, \frac{1}{2^k}<\epsilon, \forall k \geq K$ , which implies
  $|a_{n_k}-\bar{l}|< \frac{1}{2^k}<\epsilon, \forall k \geq K$
  hence $\lim_{k\to \infty} a_{n_k}=\bar{l}$ .
- Prove: $\{a_n\}$ is Cauchy $\Rarr$ $\{a_n\}$ is convergent
  The difficulty lies in formulating an explicit characterization of $l=\lim a_n$ , since Cauchiness is ostensibly defined on “relative” conditions. (Now, isn’t this fascinating? You say something “relative,” but you imply something “definite.” I think what makes this possible is not the eye-catching small relative distance part ( $|a_n-a_m|\le c\varepsilon$ ) but the quieter, $\forall n,m\ge N(\varepsilon)$ part - it allows a compression of long horizontal distances into short vertical distances, it renders potential “anchors” (like $a_N$ ), from which the tail would not sway too far.)
  Comb: Firstly, we prove that $\{a_n\}$ is bounded, then by Bolzano-Weierstrass theorem, we can have a convergent subsequence (our first grasp of $l$ ). Since if $\{a_n\}$ is convergent, its subsequences should have the same limit, we then need to prove that the limit of the subseq is indeed the limit of the original sequence.
  In definition of Cauchyness, take $\epsilon=1$ . Then there exists $N$ such that for all $n,m \geq N$ ,
  $|a_n-a_m|<1$
  Take $m=N$ , then we have $|a_n-a_N|<1 \Rightarrow |a_n|<1+|a_N|, \forall n \geq N$ .
  So after $N$ , the sequence is bounded by $1+|a_N|$ , before $N$ , there are only finite elements hence must be bounded by its maximum. $\Rightarrow \{a_n\}$ is bounded.
  Then by BW theorem, $\exist$ subsequence $\{a_{n_k}\}_{k=1}^{\infty} \to l$ . We then want to show $\lim a_n=l$ .
  Since $\lim_{k\to \infty} a_{n_k}=l$ , then $\forall \epsilon>0, \exist K(\epsilon), s.t. \forall k \geq K(\epsilon), |a_{n_k}-l|<\epsilon$ .
  Notice by triangular inequality, we can decompose $|a_n-l|$ as
  $|a_n-l|=|a_n-a_{n_k}+a_{n_k}-l|\leq |a_n-a_{n_k}|+|a_{n_k}-l|$
  The second term will be tiny given $\lim_{k\to \infty} a_{n_k}=l$ , we now use the Cauchyness to tackle the first term. Replace $m$ by $n_k$ , then from $|a_n-a_m|<\epsilon$ , we have $|a_n-a_{n_k}|<\epsilon$ . Now the key is to guarantee that $n_k \geq N(\epsilon)$ . Since $\lim_{k\to \infty} n_k=\infty$ , then $n_k \geq N(\epsilon)$ as long as $k$ is large enough (denote this threshold by $\bar{K}$ ). Consequently, as long as $k \geq max(K(\epsilon),\bar{K})$
  $|a_n-l| \leq |a_n-a_{n_k}|+|a_{n_k}-l|<2\epsilon$
  Thus $\lim_{n\to \infty} a_n=l$ .
  In reflection, this decomposition is quite intuitive. We want to characterize the relationship between $a_n$ and something definite, $l$ , knowing the relative relationship between $a_n$ ’s. Therefore we decompose $|a_n-l|$ into a term that resembles a relative relationship and a term that ties the wanderer to his Heimat. Any wanderer. Is there a subsequence of wanderers whose Heimat we know definitely exist? BW becomes a natural try.
- Prove $\limsup a_n=\sup E$
  Denote $\varlimsup a_n=\lim z_n=\bar{l}$ . Since $\bar{l} \in E$ , we have $\bar{l} \leq \sup E$ .
  Then we want to show $\bar{l} \geq \sup E$ , which is equivalent to show that $\forall l \in E, \bar{l}\geq l$ , i.e. $\bar{l}$ is larger than every element in $E$ .
  Formally, $\forall l \in E$ , by definition, there exists a subsequence whose limit is $l$ . i.e., $\exist \{a_{n_k}\}_{k=1}^{\infty},s.t.\,\,\lim_{k \to \infty} a_{n_k}=l$ .
  Notice that $z_{n_k}=\sup \{{a_{n_k},a_{n_{k+1}},a_{n_{k+2}},...}\} \geq a_{n_k}$ . $z$ 也下一楼来和它斗。
  Then $\lim z_{n_k}\geq \lim a_{n_k}$ (Rationale: $\{z_{n_k}\}_{k=1}^{\infty}$ is a subsequence of $\{z_n\}$ , since $\lim z_n=\bar{l}$ , then any of its subsequence $\{z_{n_k}\}$ converges to $\bar{l}$ ).
  Consequently, $\bar{l} \geq l, \forall l\in E$ , hence $\bar{l} \geq \sup E$ as desired.
- Qz3P1 Primitive construction of a desired sequence Back to index
  Suppose
  $\{a_n\}_{n=1}^\infty$ is not Cauchy. Prove that $\exists\epsilon_0>0$ , and subsequences $\{n_k\}_{k=1}^\infty$ and $\{m_k\}_{k=1}^\infty$ of $\{n\}_{k=1}^\infty$ , s.t.
  $1<n_1<n_2<...<n_k<...\\1<m_1<m_2<...<m_k<...$
  And $|a_{n_k}-a_{m_k}|\ge\epsilon_0,\forall k\ge 1.$
  Proof:
  $\{a_n\}_{n=1}^\infty$ is not Cauchy $\Rarr\exists\epsilon_0,\forall N,\exists m,n>N~s.t.~|a_n-a_m|\ge\epsilon_0.$ So for $N_1,\exists m_1,n_1>N_1~s.t~|a_{n_1}-a_{m_1}|\ge\epsilon_0.$ Let $N_2=max\{m_1+1,n_1+1\}\Rarr\exists m_2,n_2>N_2~s.t~|a_{n_2}-a_{m_2}|\ge\epsilon_0.$ … Let $N_k=max\{m_{k-1}+1,n_{k-1}+1\}\Rarr\exists{ m_k,n_k}>N_k~s.t~|a_{n_k}-a_{m_k}|\ge\epsilon_0.$ Continue this and we get the desired result.
  没有什么难的啦！这道题很好地展示了最难的可能就是把新seq中的后一个元素和前一个元素建立合理的联系。
- Prove the $\alpha>1$ case in Root test
  - If $\alpha=\limsup |a_n|^{\frac{1}{n}}>1$ , then $\sum |a_n|$ diverges
  - Proof
    Unlike case 1, we may not find a $N$ such that $a_n\geq \alpha$ as long as $n\geq N$ . Nevertheless, by fact 2.3.2, there exists a subsequence $\{a_{n_k}\}$ such that $\lim |a_{n_k}|^{\frac{1}{n_k}}=\alpha$ . We than want to show that $\sum |a_{n_k}|$ diverges.
    W.T.S: $|a_{n_k}|\to \infty$
    Take $\varepsilon=\frac{\alpha-1}{2}$ (to make $\alpha-\varepsilon>1$ ), then there exists $K$ such that
    $\alpha-\varepsilon<|a_{n_k}|^{\frac{1}{n_k}}<\alpha+\varepsilon, \forall k \geq K$
    Hence $|a_{n_k}|>(\alpha-\varepsilon)^{n_k}=(\frac{\alpha+1}{2})^{n_k}$
    Since $\frac{\alpha+1}{2}>1$ , $\lim_{k\to \infty} (\frac{\alpha+1}{2})^{n_k}=\infty$ , we have
    $\lim_{k\to \infty} |a_{n_k}| \geq \lim_{k \to \infty} (\frac{\alpha+1}{2})^{n_k}=\infty$
    Therefore, $\sum |a_{n_k}|$ diverges, $\sum |a_n| >\sum |a_{n_k}|$ diverges?
    Or use the divergence test, since there is a subsequence goes to infinity, $\lim a_n\neq 0$ , hence by divergence test, $\sum a_n$ diverges (这里推不出来，错的，老师上课讲的似乎有点疏漏就是最终他只证明了 $\sum a_n$ diverges ：) @The Grand Inquisitor and $\sum|a_n|$ diverges. （因为 $a_n\leq |a_n|$ , 所以由comparison test, $\sum a_n$ diverges 可以推出 $\sum |a_n|$ diverges ）行！
    But $\alpha-\epsilon>1$ , hence $\lim_{k \to \infty} (\alpha-\epsilon)^{n_k} =\infty\Rarr|a_{n_k}|\rarr\infty\Rarr\lim |a_n|\ne 0\overset{\text{div test}}{\Rarr}\sum |a_n|$ diverges. @The Grand Inquisitor 是带有绝对值的，我上面打错了我的笔记里没有绝对值，不知道是我抄错了还是老师写错了。我的笔记也没有绝对值，可能是老师写忘了。
- Hw3T1 The Happy-Face Parade Back to index
  反证假设-多米诺传导-原条件不成立, 不算严格的bad guy, 是序列内部的传导和收放，但从“传导”的角度来说还是有相似之处的。
  思路：反证，假设小于1，递推得到 $a_n>\sum_{k=0}^m\frac n{n+l+k}$ 让 m 趋于正无穷，得到 $a_n=\infty$ .
  Suppose $a_n>0,\forall n\ge 1.$ Prove that $\varlimsup_{n\rarr\infty}n(\frac{1+a_{n+1}}{a_n}-1)\ge 1.$
  Proof: (By contradiction)
  Suppose $\varlimsup_{n\rarr\infty}n(\frac{1+a_{n+1}}{a_n}-1)< 1.~\exists N\in\Z,l\in(0,1)~s.t.~n(\frac{1+a_{n+1}}{a_n}-1)<l,\forall n\ge N.$
  $\begin{aligned} a_n>\frac{n+na_{n+1}}{n+l}\Rarr & a_n>\frac n{n+l} & \\& a_{n+1}>\frac {n+1}{n+1+l}& \\a_n>\frac n{n+l}+\frac {(n+1)n}{(n+l)(n+1+l)}>\frac n{n+l}+\frac n{n+1+l} &&\\&a_{n+1}>\frac {n+1}{n+1+l}+\frac {n+1}{n+2+l}\\a_n>\frac n{n+l}+\frac n{n+1+l}+\frac n{n+2+l} \end{aligned}$ By induction we can prove $a_n>\sum_{k=0}^\infty\frac{n}{n+l+k}=\infty,\forall n>N.$
  But for any given $n,a_n$ is a finite number in a sequence and cannot be greater than $\infty$ .

先证其存，再求其值
这个比较方便的一点是，有很多东西可以用来证明极限（作为一个finite #）存在，且可能比找出确切值要容易。而一旦证明存在之后，又有很多办法来寻找极限的确切值，其中包括直接设 $l$ 然后解一元方程。
Common Ways to Prove Existence of Limit
1. MCT
1. Boundedness $\rarr$ Bolzano-Weierstrass $\rarr$ original seq also converges
  1. Proof of Cauchyness $\Rarr$ convergence.
Common Ways to Find Limit (Given Existence of Limit)
1. 得知 $l$ 一定存在之后，直接设 $l$ 然后解一元方程。很多东西都是等于 $l$ 的，比如子序列的极限，recursive form 最好解了。
- W2L1 Example 3: $\sqrt{2},\sqrt{2+\sqrt{2}},...$ does this sequence have a limit? If it does, what is it? Back to index
  Solution:
  It obviously monotonically increases. Now we prove it is bounded above by 2 using mathematical induction:
  - $a_1=\sqrt{2}<2$
  - assume $a_n<2$ , then $a_{n+1}=\sqrt{2+a_n}<2$
  Therefore $a_n<2,\forall n$ . Now by MCT, $\lim a_n$ exists as a finite number. Observe $a_{n+1}a_{n+1}=2+a_n\Rarr\lim (a_{n+1}a_{n+1})=\lim (2+a_n)\Rarr\lim a_{n+1}\lim a_{n+1}=2+\lim a_n\Rarr l^2=2+l\Rarr l=2(-1 \text{ discarded})$ .
- Does $\lim_{n\to \infty} (1+\frac{1}{n})^n$ exist? (The definition of $e$ ; proved by M.C.T.)
  Proof see here.
- Hw2 T2: Construct a sequence to approximate the root of $\alpha$ . Back to index
  - Fix a positive number $\alpha$ . Choose $x_1>\sqrt{\alpha}$ , and define $x_2,x_3,x_4,...,$ by the recursion formula $x_{n+1}=\frac{1}{2}(x_n+\frac{\alpha}{x_n})$ . Prove that $\{x_n\}$ decreases monotonically and that $\lim x_n=\sqrt{\alpha}$ .
  - Solution
    First notice that $x_{n+1}=\frac{1}{2}(x_n+\frac{\alpha}{x_n}) \geq \frac{1}{2}\times 2 \sqrt{x_n\frac{\alpha}{x_n}}=\sqrt{\alpha}$ , so $\{x_n\}$ is bounded below,
    and then $x_{n+1}-x_n =\frac{1}{2}(\frac{\alpha}{x_n}-x_n) \leq \frac{1}{2} (\frac{\alpha}{\sqrt{\alpha}}-\sqrt{\alpha})=0$ , hence decreasing. By MCT, limit exists as a finite number.
    Denote $\lim x_n=x$ , take limit on both sides of the recursive formula,
    $x=\frac{1}{2} (x+\frac{\alpha}{x}) \Rightarrow x= \pm \sqrt{\alpha}$ . Thus $\lim x_n=x=\sqrt{\alpha}$ , as the negative root is discarded.
    (b)
    Mathematical induction.
- Hw2 T3: 双生暗影 Back to index
  - Fix $\alpha>1$ . Take $x_1 > \sqrt{\alpha}$ and define $x_n=\frac{\alpha+x_n}{1+x_n}=x_n+\frac{\alpha-x_n^2}{1+x_n}$ .
  - (a) Prove that $x_1>x_3>x_5>...$ ；
    做差有点麻烦，考虑做商(注意到 $x_n>0$ )。即 $x_{n+2}=c x_n$ where $c \geq 1$ .
    $x_{n+2}=\frac{\alpha+x_{n+1}}{1+x_{n+1}}=\frac{\alpha+\frac{\alpha+x_n}{1+x_n}}{1+\frac{\alpha+x_n}{1+x_n}}=\frac{2\alpha+(\alpha+1)x_n}{1+\alpha+2x_n}=\frac{x_n (1+\alpha+ \frac{2\alpha}{x_n})}{1+\alpha+2x_n}$
    Compare $1+\alpha+\frac{2\alpha}{x_n}$ and $1+\alpha+2x_n$ ,
    if $x_n > \sqrt{\alpha},$ then $1+\alpha+\frac{2\alpha}{x_n}<1+\alpha+2x_n$ , hence $x_{n+2}<x_n$
    if $x_n < \sqrt{\alpha},$ then $1+\alpha+\frac{2\alpha}{x_n}>1+\alpha+2x_n$ , hence $x_{n+2}>x_n$
    Then all we need to show is $x_{2k+1}>\sqrt{\alpha}$ .
    Notice that $f(x_n)=\frac{\alpha+x_n}{1+x_n}$ is an decreasing function as $f'(x_n)=\frac{1-\alpha}{(1+x_n)^2}<0$ .
    Also, we have $f(\sqrt{\alpha})=\frac{\alpha+\sqrt{\alpha}}{1+\sqrt{\alpha}}=\sqrt{\alpha}$ , hence
    $f(x_n)<\sqrt{\alpha}$ iff $x_n>\sqrt{\alpha}$ and $f(x_n)>\sqrt{\alpha}$ iff $x_n<\sqrt{\alpha}$ .
    Consequently, $x_1>\sqrt{\alpha}$ , $x_2<\sqrt{\alpha}$ , $x_3>\sqrt{\alpha}$ , $x_4<\sqrt{\alpha}$ …. as desired.
  - (b) Prove that $x_2<x_4<x_6<...$ ;
    Given $x_{2k}<\sqrt{\alpha}$ , we have $x_{2k+2} > x_{2_k}$ .
  - (c) Prove that $\lim x_n=\sqrt{\alpha}$
    By MCT, both $\{x_{2k+1}\}$ and $\{x_{2k}\}$ are convergent. Denote their limits as $a$ and $b$ respectively. We then utilize the recursion formula twice (at $n+1=2k+1$ and $n+1=2k$ ) to obtain two equations about $a,b$ .
    At $n+1=2k+1$ , take limit on both sides, we have
    $\lim x_{2k+1}=\frac{\alpha+\lim x_{2k}}{1+\lim x_{2k}} \Rightarrow a=\frac{\alpha+b}{1+b}$
    Similarly, at $n+1=2k$ , we have
    $\lim x_{2k}=\frac{\alpha +\lim x_{2k+1}}{1+\lim x_{2k+1}} \Rightarrow b=\frac{\alpha+a}{1+a}$
    The solution is $a=b=\sqrt{\alpha}$ . Since both the even terms and odd terms converge to $\sqrt{\alpha}$ , the sequence converges to $\sqrt{\alpha}$ (by Hw1T4).
    我觉得TA给的存后如何得其值的方法也挺elegant，值得借鉴的：做商。
    $\frac{x_{n+2}}{x_n}=\frac{1+\alpha+\frac{2\alpha}{x_n}}{1+\alpha+2x_n}\overset{n=2k+1}{\Rarr}\lim\Rarr\frac aa=\frac{1+\alpha+\frac{2\alpha}{a}}{1+\alpha+2a}=1\Rarr a=\sqrt\alpha$ .

Cauchy Criterion - 离间计
Cauchyness 的一大好处就是可以序列内部相减相消。这对于 $a_n=A+B+C+D+...$ 这种加和样貌的序列很方便。
- Example 8: Does $\lim_{n \to \infty} (a_0+a_1q+...a_nq^n)$ exist? ( $|a_n|$ is bounded by $M$ and $|q|<1$ ) Back to index
  $\forall n\geq m\geq 1$ , we have
  $|(a_0+a_1q+...+a_nq^n)-(a_0+a_1q+...+a_mq^m)|$
  $=|a_{m+1}q^{m+1}+...+a_nq^n|$
  $\leq|a_{m+1}||q|^{m+1}+...+|a_n||q|^n$ (triangular inequality)
  $\leq M|q|^{m+1}+...+M|q|^{n}$ (boundedness of $|a_n|$ )
  $=M\frac{|q|^{m+1}-|q|^{n+1}}{1-|q|}$ (sum of geometric series)
  $\leq M\frac{|q|^{m+1}}{1-|q|}$
  We want to show $M\frac{|q|^{m+1}}{1-|q|}<\epsilon$ as long as $m$ is bigger enough. We solve for $N(\epsilon)$ .
  $M\frac{|q|^{m+1}}{1-|q|}< \epsilon$
  Take log on both sides,
  $\ln(M)+(m+1)\ln(|q|)-\ln(1-|q|)<\ln(\epsilon)$
  $\Rightarrow m > \frac{\ln(\frac{\epsilon(1-|q|)}{M})}{\ln(|q|)}-1$
  In formal proof, we need to reverse the direction, and take $N(\epsilon)=\frac{\ln(\frac{\epsilon(1-|q|)}{M})}{\ln(|q|)}-1$

1. Limit of Sequence

Definitions

不论出身，不论生平，只论终相。然无终何以论终？身之棺具体，心之椁无形，可小之又小，小小不息。

$\lim_{n\rarr\infty}a_n= l$
$\forall \varepsilon>0,\exists N(\varepsilon),s.t.~|a_n-l|<\varepsilon,\forall n\ge N(\varepsilon).$ Back to index
- Equivalently,
  1. (UF-1) What’s new: $...\le c...$
    $\forall \varepsilon>0,\exists N(\varepsilon),s.t.~|a_n-l|\le c\cdot\varepsilon,\forall n\ge N(\varepsilon)\text{ and SOME constant }c.$
    Proof of equivalence: From standard to UF-1, just take $c=1$ , the other way around, see here.
    Warning: $\varepsilon,N(\varepsilon)$ cannot appear in $c$ !!! HW1P5 wrong original solution.
  1. (UF-2) What’s new: only talk about tiny teeny $\varepsilon$ . $|a_n-l|$ 明显有最大值时考虑使用。
    $\exists \varepsilon_0>0~s.t.~\forall0<\varepsilon<\varepsilon_0,~\exists N(\varepsilon)~s.t.~|a_n-l|\le c\varepsilon~\forall x\ge N(\varepsilon).$
    Proof of equivalence: From UF-1 to UF-2, obvious cuz UF-2 only talks about a subset of UF-1’s cases. From UF-2 to UF-1, see here.
    - Example 1: Prove $\lim_{n\rarr\infty}\sqrt{1-\frac1n}=1$ . Back to index
      WTS $\forall \varepsilon>0,|1-\sqrt{1-\frac1n}|<\varepsilon. \begin{cases} \varepsilon\ge 1 & \text{always holds}\\ \varepsilon< 1 & \text{calculate }N_\varepsilon \end{cases}$
  1. (Cauchy Sequence)
    $\forall \varepsilon>0,\exists N(\varepsilon),s.t.~|a_n-a_m|\le c\varepsilon,\forall n,m\ge N(\varepsilon).$
    Proof of equivalence: “ $\Rarr$ ” here, “ $\Larr$ ” here.
    - Hw3 T2 Definition of $\{a_n\}_{n=1}^\infin$ NOT Cauchy. Back to index
      $\exists \varepsilon_0>0~s.t.~\forall N,\exists m,n>N~s.t.~|a_n-a_m|\ge\varepsilon_0.$

$\lim_{n\rarr\infty}a_n\ne l$
naive: $\exists\varepsilon_0>0,\forall N,\exists n\ge N,s.t.~|a_n-l|\ge\varepsilon_0.$
polished: $\exists \varepsilon_0>0\text{ and bad subseq }\{a_{n_k}\}^\infin_{k=1}~s.t.~|{a_{n_k}-l}|\ge\varepsilon_0,\forall k\ge1.$ Back to index
We now show the construction process of $n _1,n_2,..,n_k,...$ use the $\forall$ term
Note that he key is that both $n_k$ and $k$ goes to $\infin$ (partly by definition of subsequence).
Take $N_1=1$ , $\exists n(N_1)\geq N_1,s.t.$ $|a_{n(N_1)}-l| \geq \epsilon_0$ .
Relabel $n(N_1)$ as $n_1$ .
To guarantee $n_2 > n_1$ , we take $N_2=n_1+1$ , then there exist $n(N_2) \geq N_2$ such that $|a_{n(N_2)}-l| \geq \epsilon_0$ . Relabel $n(N_2)$ as $n_2$ and we know $n_2 \geq N_2 >n_1$ .
Repeat this process, we have bad subseq $\{a_{n_k}\}^\infin_{k=1}$ .

$\lim_{n\rarr\infty}a_n= \infty$
$\forall M>0,\exists N(M)~s.t.~a_n>M~\forall n\ge N(M).$

$\nexists\lim_{n\rarr\infty}a_n$
第一个是极限存不存在的问题，等于一个 finite number $l$ 或者等于 $\pm \infty$ ，都算是极限存在。极限不存在使用 $\lim a_n$ 就不是well-defined，所以我觉得 $\forall l, \lim a_n \neq l$ 也不是一个很严谨的说法，因为极限不存在的话是不能用 $\lim a_n$ 这个符号的（就这样说有点默认 $\{a_n\}$ 的极限存在了，只是不等于 $l$ 罢了）。在 $(-1)^n$ 的例子里，说它极限不存在我们也没有严谨证明。第二个问题是极限存在后，这个极限是不是有限的。
$\epsilon-N$ 这种定义是把是否convergent和converge到哪里混合在一起的，cauchy的定义就只涉及convergent，但不说converge到哪里。所以我想到的极限不存在的说法是 1. 不是柯西的（这样它是divergent）+ 2. 不趋于 $\pm \infty$ .
行吧，应该不是big issue. 估计这几个terminologies都没有很统一，符号的意思大概比语言的意思更本质。

Basic Facts

1.1 收敛必有界 $\{a_n\} \text{ is convergent}\Rarr \{a_n\}\text{ is bounded}$
断点法首秀。点之前利用finite # of #取max，点之后利用极限的定义和triangle inequality.
Let $\lim a_n=l\Rarr\text{ take }\varepsilon=411,\exists N(411) ~s.t.~|a_n-l|\le\varepsilon=411,\forall n\ge N(411)\Rarr|a_n|-|l|\le|a_n-l|\le411\Rarr|a_n|\le411+|l|,\forall n\ge N(411).$
As for $n<N(411),\text{ take }M=max\{411+|l|,|a_1|,|a_2|,...,|a_{N(411)-1}|\}\Rarr|a_n|\le M, \forall n\ge 1.\blacksquare$
Remark: $\{a_n\} \text{ is bounded}\nRightarrow \{a_n\}\text{ is convergent}$

1.2 Squeezing Theorem: Suppose $a_n\le b_n\le c_n~\&~\lim a_n=\lim c_n=l\Rarr\lim b_n\text{ exists and}=l$
只要足够大……那什么才叫足够大？ $\Rarr$ 找 $\max\{N_1,N_2,...\}$ .
Proof: $\because\lim a_n=\lim c_n=l\therefore a_n,c_n\in(l-\varepsilon,l+\varepsilon),\forall n\ge \max\{N_a(\varepsilon),N_c(\varepsilon)\}\Rarr b_n\in(l-\varepsilon,l+\varepsilon),\forall n\ge \max\{N_a(\varepsilon),N_c(\varepsilon)\}.\blacksquare$

1.3 四则运算：Suppose $\lim a_n=a,\lim b_n=b$
- $\lim (a_n\pm b_n)=\lim a_n\pm\lim b_n$ (证明思路同1.2)
- $\lim(ca_n)=c\lim a_n$
- $\lim a_nb_n=\lim a_n\lim b_n$
  Proof hint: $a_nb_n-ab=(a_n-a)(b_n-b)+a(b_n-b)+b(a_n-a)$
- $\lim \frac{a_n}{b_n}=\frac{\lim a_n}{\lim b_n},\text{ if }\lim b_n\ne 0$
  Proof: see here.

1.4 极限与序列元素相对大小一致: Suppose $\lim a_n=a,\lim b_n=b,a_n\le b_n$ for $n\ge\text{ some }N,\text{ then }a\le b.$
Proof: (By contradiction) Suppose $a>b$ , we try to create a circumstance under which $a_n>b_n$ for large enough $n.$
$\forall \varepsilon>0, \exists N~s.t.~a-\varepsilon<a_n<a+\varepsilon,b-\varepsilon<b_n<b+\varepsilon,\forall n>N$ . Dream: $b+\varepsilon<a-\varepsilon\Rarr\varepsilon<\frac{a-b}2.$ So we take $\varepsilon= \frac{a-b}{411},\text{ then }a_n<a+\varepsilon<b-\varepsilon<b_n\text{ if }n\ge N(\frac{a-b}{411}). \blacksquare$
Corollary:
If $a_n\ge m,\forall n\ge N,\text{ then if }\exists\lim a_n,\lim a_n\ge m.$
However, $"a_n> m,\forall n\ge N,\text{ then if }\exists\lim a_n,\lim a_n> m"$ is WRONG.

1.5 子列与原列极限一致 Suppose $\lim a_n=l,$ Then for any subsequence $\{a_{n_k}\}^\infty_{k=1}$ , we have $\lim a_{n_k}=l.$
Proof see here.

1.6 原列与子子列极限一致 Suppose $\{a_n\}$ satisfies: $\forall$ subsequence of $\{ a_n\}$ , $\exist$ sub-subsequence $\{a_{n_k}\}_{k=1}^{\infty}$ , s.t. $\lim_{k\to \infty} a_{n_k}$ exists & $=l$ , where $l$ is independent of the choice of subsequence. Then $\lim_{k\to \infty} a_{n}$ exists & $=l$ .
Proof see here.
Remark: by 1.6, the converse of 1.5 holds: if for any subsequence $\{a_{n_k}\}^\infty_{k=1}$ , we have $\lim a_{n_k}=l,$ then $\lim a_n=l.$
1.6 is more general than the converse of 1.5, because it only requires each subsequence to contain one ( $\exists$ ) subsubseq that converge to $l$ , whereas the converse of 1.5 says any subseq itself should converge to $l$ . Note that $\exist$ sub-subsequence $\{a_{n_k}\}_{k=1}^{\infty}$ , s.t. $\lim_{k\to \infty} a_{n_k}$ exists & $=l$ , doesn’t necessarily give limit of subseq also exists and is $l$ (e.g., $(-1)^n$ ). However, if we include the $\forall$ part into our consideration, is 1.6 really more general than the converse of 1.5? Can we find a seq where “ $\forall$ subsequence of $\{ a_n\}$ , $\exist$ sub-subsequence $\{a_{n_k}\}_{k=1}^{\infty}$ , s.t. $\lim_{k\to \infty} a_{n_k}$ exists & $=l$ , where $l$ is independent of the choice of subsequence” is satisfied yet “ $\forall$ subsequence $\{a_{n_k}\}^\infty_{k=1}$ , we have $\lim a_{n_k}=l$ ” is not? @The Grand Inquisitor
@Cindy Zeng 我觉得没有，因为如果前面的条件满足，那么 $\lim a_n$ exists & $=l$ ，然后由fact 5就可以推出后面那个条件。所以这两个条件可以互相imply，它们应该是等价的。
@The Grand Inquisitor 我也觉得没有更general…

(1.7) Monotone Convergence Theorem
Suppose $\{a_n\}$ is monotone & bounded, then $\lim_{n\rarr\infty}a_n~\exists\text{ as a finite } \#.$
- Supremum and Infimum
  - A real # $s$ is said to be the “least upper bound (supremum 上确界)” of $S$ if
    1. $s$ is an upper bound of $S$ , i.e., $\forall x\in S, x\le s.$
    1. $\forall \varepsilon>0,s-\varepsilon<x_\varepsilon \text{ for some }x_\varepsilon\in S.$
    Intuitively, either $s=\max S$ or elements of $S$ 在 $s$ 之下紧密排布，“没有最近，总有更近的。”
  - A real # $s$ is said to be the “greates lower bound (infimum 下确界)” of $S$ if
    1. $s$ is an lower bound of $S$ , i.e., $\forall x\in S, x\ge s.$
    1. $\forall \varepsilon>0,s+\varepsilon>x_\varepsilon \text{ for some }x_\varepsilon\in S.$
  Remark: $\forall$ upper bound $M$ of $S,$ we have $M\ge\sup S$ ; $\forall$ lower bound $m$ of $S,$ we have $m\le\inf S$ .
  - Example: Hw2 T1 Back to index
    (iv) rational number: can be represented by $\frac pq$ where p, q are integers. 思路和证明极限等于某某差不多。
- The Least Upper Bound Axiom (公理)
  If $S$ is bounded from above, then $\sup S$ exists. 总存在最小的上界。
  $\rarr$ Theorem: If $S$ is bounded from below, then $\inf S$ exists.
  Proof: Let $T=-S\Rarr T$ is bounded from above. By axiom, $\sup T$ exists as a finite number, and it satisfies
  1. $\forall -s\in T,\text{ we have }-s\le \sup T\Rarr s\ge-\sup T,\forall s\in S$
  1. $\forall \varepsilon>0,\exists -s_\varepsilon\in T~s.t.~\sup T-\varepsilon<-s_\varepsilon\Rarr s_\varepsilon<-\sup T+\varepsilon\Rarr -\sup T \text{ is }\inf S.$
  By-product: $\sup(-S)=-\inf S;-\sup(S)=\inf (-S)$ 名字相对，负号两次
- Proof of MCT
  Only consider the case of increasing $a_n$ . $\{a_n\}$ is bounded from above $\overset{\text{Axiom}}{\Rarr}\sup\{a_n\}_{n=1}^\infty$ exists, we let it be $l$ . By definition:
  1. $a_n\le l,\forall n~\bigstar$
  1. $\forall \varepsilon>0,l-\varepsilon<a_{N_\varepsilon}\text{ for some }N_\varepsilon.$ $a_n\uarr~\Rarr l-\varepsilon<a_{N_\varepsilon}\le a_n\overset{\bigstar}{\le}l<l+\varepsilon,\forall n\ge N_\varepsilon\Lrarr |a_n-l|<\varepsilon\Rarr \lim a_n=l. \blacksquare$ 会给挤压到天花板上。
  Remark: If $a_n\uarr$ & not bounded from above, $\lim a_n\rarr\infty$ (you’re going to heaven); If $a_n\darr$ & not bounded from below, $\lim a_n\rarr-\infty$ (you’re going to hell). Meaning if we adopt the conceptual framework described in the beginning of this note, monotonicity $\Rarr$ limit must exist (either as a finite number or as infinities).
- Example
  Example 4: Does $\lim_{n\to \infty} (1+\frac{1}{n})^n$ exist? (The definition of $e$ ; proved by M.C.T.)
  Solution see here.

2. Upper and Lower Limits

Motivation

$\lim_{n\rarr\infty} a_n$ does not always exist, but upper and lower limites always exist. And we can use them to prove the existence of / directly find $\lim_{n\rarr\infty} a_n$ . Prof. Wang loves it. Powerful stuff.

Definition

Consider $\{a_n\}^\infty_{n=n_0}$ , WLOG, assume $n_0=1$ for convenience (we don’t really care about $n_0$ ).

$y_1=\inf\{a_1,a_2,...,a_n,...\}~^*\pm\infty\text{ allowed.}\\ y_2=\inf\{a_2,a_3,...,a_n,...\}\\y_n=\inf\{a_n,...\}$

And in this way we construct a new sequence $\{y_n\}_{n=1}^\infty$ and $y_n\uarr$ . By MCT, $\lim_{n\rarr\infty} y_n$ exists and may be $\pm\infty$ . We call $\lim_{n\rarr\infty} y_n$ the “lower limit” of $\{a_n\}$ , and we denote it as $\lim_{n\rarr\infty}\inf a_n$ or $\varliminf_{n\rarr\infty} a_n$ .

Similarly, we can construct a new sequence $\{z_n\}_{n=1}^\infty$ and $z_n\darr$ , where $z_n=\sup\{a_n,a_{n+1,...}\}$ . We call $\lim_{n\rarr\infty} z_n$ the “upper limit” of $\{a_n\}$ , and we denote it as $\lim_{n\rarr\infty}\sup a_n$ or $\varlimsup_{n\rarr\infty} a_n$ .

Basic Facts

2.1 下极限小于上极限 $\varliminf_{n\rarr\infty} a_n\le \varlimsup_{n\rarr\infty} a_n$
Proof: $y_n\le z_n\overset{F1.4}{\Rarr}\lim y_n\le \lim z_n. \blacksquare$

2.2 中原存则蛮夷附之 $\lim_{n\rarr\infty} a_n$ exists (allow $\pm\infty$ ) $\Lrarr\varliminf_{n\rarr\infty} a_n=\varlimsup_{n\rarr\infty} a_n(=\lim_{n\rarr\infty} a_n)$
Proof:
“ $\Rarr$ ”.
case 1: $\lim_{n\rarr\infty} a_n=l$ (finite #) $\Rarr\forall\varepsilon>0,\exists N~s.t.~l-\varepsilon<a_n<l+\varepsilon,\forall n\ge N\Rarr l-\varepsilon\le y_n\le z_n\le l+\varepsilon,\forall n\ge N\Rarr \lim y_n=l=\lim z_n.$
case 2: $\lim_{n\rarr\infty} a_n=\infin\Rarr \forall M>0,\exists N~s.t.~a_n\ge M,\forall n\ge N\Rarr y_n\ge M\Rarr \lim_{n\rarr\infty} y_n=\infty\overset{F2.1}{\Rarr}\varliminf_{n\rarr\infty} a_n= \varlimsup_{n\rarr\infty} a_n=\infty$
case 3: $-\infty$ . Similar to case 2. $\blacksquare$
“ $\Larr$ ”.
case 1: $\lim_{n\rarr\infty} a_n=l$ (finite #) $\Rarr\forall\varepsilon>0,\exists N~s.t.~l-\varepsilon<y_n,z_n<l+\varepsilon,\forall n\ge N\Rarr l-\varepsilon\le y_n\le a_n\le z_n\le l+\varepsilon,\forall n\ge N\Rarr \lim a_n=l.$
case 2: $\lim_{n\rarr\infty} y_n=\infin\Rarr \forall M>0,\exists N~s.t.~y_n\ge M,\forall n\ge N\Rarr a_n\ge M\Rarr \lim_{n\rarr\infty} a_n=\infty.$
case 3: $-\infty$ . Similar to case 2. $\blacksquare$

2.3.1 上下极限之外，远则不遇 $\text{If }a_0>\varlimsup_{n\rarr\infty} a_n,\text{ then }\exists N~s.t.~a_0>a_n,\forall n\ge N\\~~~~~~~~\text{If }b_0<\varliminf_{n\rarr\infty} a_n,\text{ then }\exists N~s.t.~b_0<a_n,\forall n\ge N$
2.3.2
总存在收敛到上极限/下极限的子序列 Define a set $E=\{a|\exists \text{ subseq }a_{n_k}\text{ with limit } a,\text{ allow }a\text{ to be }\pm\infty.\}\text{ Then }\varliminf_{n\rarr\infty} a_n, \varlimsup_{n\rarr\infty} a_n\in E$ .
2.3.3
（上）下极限是E的（上）下确界 $\varliminf_{n\rarr\infty} a_n=\inf E;~\varlimsup_{n\rarr\infty} a_n=\sup E.$
Proofs:
2.3.1
$\lim_{n\rarr\infty} z_n=\bar l\Rarr\forall\varepsilon>0,\exists N~s.t.~\text{if }n\ge N,a_n\le z_n<\bar l+\varepsilon.\text{ Take } \varepsilon=a_0-\bar l\Rarr a_n<a_0,\forall n\ge N.$
Rest similar.
2.3.2
See here.
2.3.3
By Fact 2.3.2 we know that $\varlimsup_{n\rarr\infty} a_n\in E\Rarr\varlimsup_{n\rarr\infty} a_n\le\sup E.\text{ WTS }\forall a\in E, \varlimsup_{n\rarr\infty} a_n\ge a (\bigstar).$ By definition of $E,\exists \text{ subseq }\{a_{n_k}\}_{k=1}^\infty~s.t.~a_{n_k}\rarr a.$ $\because \lim z_n=\bar l\therefore\lim z_{n_k}=\bar l.~z_{n_k}\ge a_{n_k}\overset{F1.4}{\Rarr}\bar l\ge a,\forall a\in E\Rarr \bar l\ge \sup E\overset{\bigstar}{\Rarr}l= \sup E. \blacksquare$

2.4 上下极限与序列元素相对大小一致（类比F1.4） $\text{If }a_n\le b_n,\forall n\ge\text{ some }N,\text{ then }\varliminf_{n\rarr\infty} a_n\le \varliminf_{n\rarr\infty} b_n,\varlimsup_{n\rarr\infty} a_n\le \varlimsup_{n\rarr\infty} b_n.$
Proof: @The Grand Inquisitor 小天，帮我看看这个证明对不对😭😫
Case 1: $\varliminf_{n\rarr\infty} b_n$ is a finite number.
Suppose $\varliminf_{n\rarr\infty} a_n> \varliminf_{n\rarr\infty} b_n$ , by Fact 2.3.1, $\exists N_1~s.t.~\lim_{n\rarr\infty} y^b_n<a_n,\forall n\ge N_1$ , take $\epsilon=\frac{a_{N_1}-\lim_{n\rarr\infty} y^b_n}2,\text{ then }y_{N_1}^b<\lim_{n\rarr\infty} y^b_n+\epsilon<a_{N_1}$ . Let $N_2=N_1+1,\epsilon_2=...$ we can construct a bad sequence of $\{y^b_{N_n}\}_{n=1}^\infty$ where $y^b_{N_n}<a_{N_n},\forall n\ge 1,$ contradicting $a_n\le b_n,\forall n\ge\text{ some }N.$ 到红色为止就可以结束了，前面加一个 $b_{N_1}<y_{N_1}^b$ , 就能得出 $b_{N_1}<a_{N_1}$ 的矛盾（这个只是在某个值上面满足，似乎不够）
我当时想了想好像不行，我们需要一个完整的序列来contradict $\forall n\ge$ some N 那个部分，因为 $N_1$ 不行，或许比他大的some N 还是可以work的。嗯你说的对！本来以为真就是F1.4的翻版，一试发现似乎没那么简单O.o#.
我觉得利用 $y_n^a\leq y_n^b$ (集合中每个元素都更小，inf也更小)，让然后就可以直接用fact 1.4 说 $\lim y_n^a\leq \lim y_n^b$ ，也即 $\underline{\lim} a_n \leq \underline{\lim} b_n$ @The Grand Inquisitor 天，我突然想到一个问题，我们并不知道 $\lim y_n^a, \lim y_n^b$ 存在啊？(@Cindy Zeng 根据MCT， $\lim y_n$ 和 $\lim z_n$ 都存在，所以说liminf 和 limsup总是有的)
诶，有道理。别走 @The Grand Inquisitor 可是那我那个复杂的方法对不对啊，感觉每次取一个不同的 $\epsilon$ 很心虚。上次好像就错过一次，因为不能反复取 $\epsilon$ .
这里没问题，极限不存在和不cauchy的例子不能反复取不同的 $\varepsilon$ 是因为都是针对的同一个bad $\varepsilon$ ，但像证明有一个子序列极限为limsup的时候，第n轮的 $\varepsilon_n=\frac{1}{n}$ ，每轮都是递减的。
嗯对，有时可以，有时不行。可能就要注意分清什么时候要一 $\varepsilon$ 到底，什么时候只需要找到一个它就好了（可能当 $\forall \varepsilon$ 是条件的一部分的时候，找到一个就行，但当我们需要说明 $\exists\varepsilon_0$ balabala 的时候就必须一 $\varepsilon$ 到底）。主要是之前我错的那个地方我也记不太清了呜呜，好像是某道作业题，没事，应该找得到。

2.5 (Useful trick) $\lim a_n=l\Lrarr\varlimsup_{n\rarr\infty} |a_n-l|=0.$ 很有用！尤其是当我们不知道lim是否存在的时候。
$0\le \varliminf|a_n-l|\le \varlimsup|a_n-l|=0\Rarr\lim|a_n-l|=0.$

2.6 合租房比整租房扁 $\varlimsup_{n\rarr\infty} (a_n+b_n)\le\varlimsup_{n\rarr\infty} a_n+\varlimsup_{n\rarr\infty}b_n\\~~~~~\varliminf_{n\rarr\infty} (a_n+b_n)\ge\varliminf_{n\rarr\infty} a_n+\varliminf_{n\rarr\infty}b_n$ (True as long as $\infin-\infin$ does not appear on RHS.)
Proof (Hw2T4.2, 4.3): Back to index
规整和上确界≤打乱和上确界≤分开上确界（让我想起高中作文素材曾摘了一个形容独立性的重要性的美句哈哈哈，！找到了！“廊柱分立才能撑起庙宇。橡树和松柏也不能在彼此的阴影中生长。”——纪伯伦《先知》顺便看到了一句无关的，分享分享：“有人曾问过王劲松，荀彧为什么会相信曹操可以帮他匡扶汉室?王劲松用剧中的一句词作答:万古长夜当中,哪怕有一盏微弱的光芒，都会让你身不由己追随，至死方休。” @The Grand Inquisitor ）军师联盟😭😭😭其实这个(包括下面那个)的证明思路有点像minmax。什么minmax? 为什么我第一反应是信息竞赛里的什么东西？？？证明零和博弈纳什均衡的一个定理
我不知道这个😓，但是我第一眼看到你这个证明的时候就觉得这个思路很有前途😁。
Let $A_n=\{a_n,a_{n+1},...\},B_n=\{b_n,b_{n+1},...\},S_n=\{a_n+b_n,a_{n+1}+b_{n+1},...\},SS_n=\{a_i+b_j|i\ge n,j\ge n,i,j\in\Z\}. \forall a_i+b_j\in SS_n,a_i+b_j\le \sup(A_n)+\sup(B_n)\Rarr\sup(SS_n)\le \sup(A_n)+\sup(B_n).$
$\because S_n\sub SS_n\therefore\forall a_i+b_i\in S_n,a_i+b_i\in SS_n \therefore a_i+b_i\le\sup(SS_n)\Rarr\sup(S_n)\le \sup(SS_n)\Rarr\sup(S_n)\le \sup(A_n)+\sup(B_n)\Rarr\varlimsup_{n\rarr\infty} (a_n+b_n)\le\varlimsup_{n\rarr\infty} a_n+\varlimsup_{n\rarr\infty}b_n$ .

2.7 脱帽无妨 If $\lim_{n\rarr\infty} a_n$ exists as a finite #,

$\varlimsup_{n\rarr\infty} (a_n+b_n)=\lim_{n\rarr\infty} a_n+\varlimsup_{n\rarr\infty}b_n\\\varliminf_{n\rarr\infty} (a_n+b_n)=\lim_{n\rarr\infty} a_n+\varliminf_{n\rarr\infty}b_n$
Proof (Hw2T4.4): Back to index
还是关于上下极限和极限三者分析的老思路：筷子都夹住了，中间必有汤圆。
Preserve the notations used in proving Fact 2.6. $\lim a_n=\varliminf a_n=\varlimsup a_n~(\star).$ $\forall a_i+b_i\in S_n,i.e.,i\ge n,~\sup(S_n)\ge a_i+b_i\ge\inf(A_n)+\sup(B_n)\Rarr \inf(A_n)+\sup(B_n)\le \sup(S_n)\le \sup(A_n)+\sup(B_n)\Rarr \varliminf_{n\rarr\infty} a_n+\varlimsup_{n\rarr\infty} b_n\le \varlimsup_{n\rarr\infty} (a_n+b_n)\le \varlimsup_{n\rarr\infty} a_n+\varlimsup_{n\rarr\infty}b_n\overset{\star}{\Rarr}\varlimsup_{n\rarr\infty} (a_n+b_n)= \lim_{n\rarr\infty} a_n+\varlimsup_{n\rarr\infty}b_n$ .

2.8 首尾颠倒，负号两次 $\varlimsup_{n\rarr\infty} (-a_n)=-\varliminf_{n\rarr\infty} a_n, \varliminf_{n\rarr\infty} (-a_n)=-\varlimsup_{n\rarr\infty} a_n$
Proof (Hw2T4.1): Back to index
Let $S_n=\{a_n,a_{n+1},...\},-S_n=\{-a_n,-a_{n+1},...\},z_n=\sup(-S_n),y_n=\inf(S_n)$ , by the byproduct of the least upper bound axiom, $z_n=-y_n\Rarr\varlimsup_{n\rarr\infty}(-a_n)=\lim_{n\rarr\infty} z_n=-\lim_{n\rarr\infty} y_n=-\varliminf_{n\rarr\infty} a_n$ .

Theorem: Bolzano-Weierstrass Theorem
If $\{a_n\}$ is bounded, then it contains a convergent subseq.
Proof:
$\{a_n\}$ is bounded $\Rarr\varlimsup a_n$ is finite. $\varlimsup a_n\in E\Rarr\exists$ a subseq converging to $\varlimsup a_n$ .

3. Series

Definition: Given a sequence of

\{a_n\}

, we call

$\sum_{n=n_0}^{\infty} a_n$ an infinite series

$s_n=\sum_{k=n_0}^{n} a_k$ partial sums of the sequence

$\{s_n\}$ constitutes a new sequence: if $\lim_{n\to \infty} s_n$ exists as a finite number, then we say $\sum_{n=n_0}^{\infty} a_n$ converges and define $\sum_{n=n_0}^{\infty} a_n=\lim_{n\to \infty} s_n$ . Otherwise, we say $\sum_{n=n_0}^{\infty} a_n$ diverges.

Basic Facts

3.0 开端不影响结局，殊始同归 $\sum_{n=1}^{\infty} a_n$ and $\sum_{n=n_0}^{\infty} a_n$ have the same convergence/divergence nature. (although the value may be different).

3.1 $\colorbox{yellow}{Divergence Test}$  最后增量需趋于零，天下归心: If $\sum a_n$ converges, then $\lim_{n \to \infty} a_n=0$ . Consequently, if $\lim_{n \to \infty} a_n \neq 0$ . then $\sum a_n$ diverges. ( $\lim_{n \to \infty} a_n=0$ 是 $\sum a_n$ converge 的必要不充分条件).
Proof: Observe $a_n=s_n-s_{n-1}$ . Take limit on both sides:
$\lim a_n=\lim (s_n-s_{n-1})=\lim s_n-\lim s_{n-1}=0$ .

3.2 只要足够远，绝对片面和就足够小 $\sum a_n$ converges $\Leftrightarrow$ $\forall \epsilon>0, \exists N, s.t.$ $n,m \geq N$ (WLOG, $n\geq m$ ), $|\sum_{k=m}^n a_k|< \epsilon$ .
- Proof: $\sum a_n$ converges $\Leftrightarrow \lim s_n$ exists & finite $\overset{\text{Cauchy's Criterion}}{\Leftrightarrow} \{s_n\}$ is Cauchy.
  $\forall \epsilon>0, \exist N, s.t. \forall n,m \geq N, | s_n-s_{m-1}|=| \sum_{k=m}^{n} a_k |< \epsilon$
- Remark 整条尾巴都要足够扁 remainder shrinks to 0: If $\sum a_n$ converges, then $\sum_{n=k}^{\infty} a_n \to 0$ as $k \to \infty$ .
  Proof: $\sum_{n=k}^{\infty} a_k= \sum_{n=1}^{\infty} a_n-s_{k-1}$ . Take limits on both sides of the equation
  $\lim_{k \to \infty} \sum_{n=k}^{\infty} a_k =\sum_{n=1}^{\infty} a_n-\lim_{k\to \infty} s_{k-1}=\lim_{n \to \infty} s_n-\lim_{n\to \infty }s_n=0$
  Because $\sum_{n=1}^{\infty} a_n=\lim_{n \to \infty}$ is just a finite number
- Example: 尾巴扁不下来
  - Hw3 T10(b): Suppose $a_n>0$ , $S_n=a_1+...+a_n$ and $\sum a_n$ diverges. Prove that $\frac{a_{N+1}}{S_{N+1}}+...+\frac{a_{N+k}}{S_{N+k}}\geq 1-\frac{S_N}{S_{N+k}}$ and deduce that $\sum \frac{a_n}{s_n}$ diverges. Back to index
    Note that
    $1-\frac{S_N}{S_{N+k}}=\frac{S_{N+k}-S_N}{S_{N+k}}=\frac{a_{N+1}+...+a_{N+k}}{S_{N+k}} \leq \frac{a_{N+1}}{S_{N+1}}+...+\frac{a_{N+k}}{S_{N+k}}$
    The last inequality holds because $a_n>0$ hence $s_n$ is increasing.
    To prove that $\sum \frac{a_n}{S_n}$ diverges, we verify the opposite of its convergent conditions
    Take $\varepsilon=\frac{1}{2}$ , then $\forall N$ , we take $n=N+1, m=N+k$ , so that
    $|\sum_{j=N+1}^{N+k} a_j|=|\frac{a_{N+1}}{S_{N+1}}+...+\frac{a_{N+k}}{S_{N+k}}| \geq 1-\frac{S_N}{S_{N+k}}$
    Since $\lim S_n =\infty$ . For any fixed $N$ , we have $\lim_{k\to \infty} \frac{S_N}{S_{N+k}}=0$ , so we can find a $k$ big enough, such that $\frac{S_N}{S_{N+k}}<\frac{1}{2}$ . This means $|\sum_{j=N+1}^{N+k}a_j|\geq \frac{1}{2}$ .
  - Hw3 T11(a): Back to index
    $\frac{a_m}{r_m}+...+\frac{a_n}{r_n}>\frac{a_m+...+a_n}{r_m}=\frac{r_m-r_{n+1}}{r_m}=\frac{r_m-\colorbox{greenyellow}{$r_n+r_n$}-r_{n+1}}{r_m}=\frac{r_m-r_n+a_n}{r_m}>\frac{r_m-r_n}{r_m}=1-\frac{r_n}{r_m}.$ Since $\sum a_n$ converges, by remark of F3.2, $r_n\rarr\infty$ as $n\rarr\infty$ $\Rarr\forall m,\exists n_0>m~s.t.~r_{n_0}\le \frac{r_m}2\Rarr\frac{a_m}{r_m}+...+\frac{a_{n_0}}{r_{n_0}}>1-\frac{r_{n_0}}{r_m}\ge \frac12\Rarr\sum\frac{a_n}{r_n}$ is not Cauchy, therefore it diverges.
    For Cauchy stuff, you can tinker with $m,n$ since they are behind $\forall.$

3.3 Absolute Convergence 左边更难conv（所以更强），因为取和的话绝对值消除了正负相抵的可能性: $\sum_{n=1}^{\infty} |a_n|$ converges $\Rightarrow$ $\sum_{n=1}^{\infty} a_n$ converges.
Proof: By Fact 2, $\forall \epsilon>0, \exist N, s.t. \forall n,m\geq N,$
$|\sum_{k=m}^{n} a_k| \leq \sum_{k=m}^n |a_k|< \epsilon$ (By Triangular Inequality)
Then by Fact 2 again, $\sum_{n=1}^{\infty} a_n$ converges.
Compare: F1.2: convergence of $\{a_n\}\Rarr$ convergence of $\{|a_n|\}$ but not the reverse. 因为不取和则绝对值更易converge，它可以把和两岸的人团结成一股力量（Der Mensch ist ein Seil über einem Abgrunde.)

3.4 Comparison Test: Suppose $0\leq a_n \leq b_n, \forall n \geq$ some $n_0$ , then
- 内收 If $\sum b_n$ converges, then so does $\sum a_n$ .
- 外推 If $\sum a_n$ diverges, then so does $\sum b_n$ .
- Proof
  (i) WLOG, assume $n_0=1$ .
  $a_n \leq b_n \Rightarrow s_n^{a} \leq s_n^{b}$ . Moreover, since $\sum b_n$ converges, $\lim s_n^{b}$ exists as a finite number, then $\{s_n^{b}\}$ is bounded, denote the bound by $M$ . Therefore, $s_n^{a} \leq s_n^{b} \leq M$ .
  In addition, $a_n \geq 0$ , $s_n^{a}$ is monotone increasing and bounded below by 0. Consequently, $\{s_n^{a}\}$ is monotonically increasing and bounded, by MCT, $\lim s_n^{a}$ exists as a finite number, and thus $\sum a_n$ converges.
  (ii) Since $\sum a_n$ diverges and $\{ s_n^{a}\}$ is monotonically increasing, by MCT, we have $\lim s_n^a=\infty$ . Then $\lim s_n^b =\infty$ and $\sum b_n$ diverges.
- Remark: If $a_n\ge0$ , then $\sum a_n$ either converges to a finite number or $\sum a_n=\infin$ .
- Examples: 常见锚点p-series/geometric series/telescope sum, i.e., $\sum \frac{1}{x^p},\sum \beta^n$ and $\sum (x_n-x_{n-1})$ .
  - Hw3 T8 (a): $a_n=\sqrt{n+1}-\sqrt{n}$ Back to index
    Multiplying and dividing $a_n$ by $\sqrt{n+1}+\sqrt{n}$ , we find that
    $a_n=\frac{1}{\sqrt{n+1}+\sqrt{n}}>\frac{1}{2\sqrt{n+1}}$
    Since $\sum \frac{1}{\sqrt{n}}$ diverges ( $p=\frac{1}{2}<1$ ), hence $\sum a_n$ diverges
    Alternatively, since the sum telescopes, the $n$ th partial sum $s_n=\sqrt{n+1}-1$ , which obviously tends to infinity.
  - Hw3 T8 (b): $a_n=\frac{\sqrt{n+1}-\sqrt{n}}{n}$ Back to index
    Using the same trick above, we find that $a_n=\frac{1}{n[\sqrt{n+1}+\sqrt{n}]}<\frac{1}{2 n^{\frac{3}{2}}}$ .
    Since $\sum \frac{1}{n^{\frac{3}{2}}}$ converges ( $p=\frac{3}{2}>1$ ), by comparison test, $\sum a_n$ converges
  - Hw3 T9: Let $a_n \geq 0$ , prove that the convergence of $\sum a_n$ implies the convergence of $\sum \frac{\sqrt{a_n}}{n}$ [Hint: Comparison test and $|ab|\leq \frac{a^2+b^2}{2}$ ] Back to index
    Note that $\frac{\sqrt{a_n}}{n}=\sqrt{a_n}\times \frac{1}{n}\leq \frac{(\sqrt{a_n})^2+(\frac{1}{n})^2}{2}=\frac{1}{2}(a_n+\frac{1}{n^2})$
    Since $\sum a_n$ converges, $\sum \frac{1}{n^2}$ converges, we have $\sum (a_n+\frac{1}{n^2})$ converges.
    By comparison test, $\sum \frac{\sqrt{a_n}}{n}$ converges
  - HW3 T10 (c): Suppose $a_n>0$ , $S_n=a_1+...+a_n$ and $\sum a_n$ diverges. Prove that $\frac{a_n}{s_n^2}\leq \frac{1}{s_{n-1}}-\frac{1}{s_n}$ and deduce that $\sum \frac{a_n}{s_n^2}$ converges. Back to index
    $\frac{1}{s_{n-1}}-\frac{1}{s_n}=\frac{s_n-s_{n-1}}{s_{n-1}s_n}=\frac{a_n}{s_{n-1}s_n} \geq \frac{a_n}{s_n^2}$
    Denote $ss_n=\sum_{k=2}^{n} (\frac{1}{s_{k-1}}-\frac{1}{s_k})=\frac{1}{s_1}-\frac{1}{s_n}$ . Hence
    $\lim ss_n=\frac{1}{s_1}-\lim \frac{1}{s_n}=\frac{1}{s_1}$
    Therefore, $\sum (\frac{1}{s_{n-1}}-\frac{1}{s_n})$ converges. By comparison test, $\sum \frac{a_n}{s_n^2}$ converges.
  - Hw3 T10 (d): Construct a sequence $\{a_n\}$ such that $\sum a_n$ diverges but $\sum \frac{a_n}{1+na_n}$
    Let $a_n=\frac{1}{n^2}$ if $n$ is not a perfect square and $a_n=\frac{1}{\sqrt{n}}$ if $n$ is a perfect square (the bad guy). Back to index
    $\sum a_n \geq \sum_{n=k^2} \frac{1}{k}=\sum_{k=1}^{\infty} \frac{1}{k}=\infty$ , hence $\sum a_n$ diverges
    But $\sum \frac{a_n}{1+na_n} = \sum_{n\neq k^2} \frac{1}{n^2+n^3}+\sum_{n=k^2} \frac{1}{k+k^2} \leq \sum_{n=1}^{\infty} \frac{1}{n^2+n^3}+\sum_{k=1}^{\infty}\frac{1}{k+k^2}$
    Both $\sum_{n=1}^{\infty} \frac{1}{n^2+n^3}$ and $\sum_{k=1}^{\infty} \frac{1}{k+k^2}$ converges, hence $\sum \frac{a_n}{1+na_n}$ converges.
    @Cindy Zeng 你觉得这个地方的rearrangement 可能会有问题嘛，就把能开平方数和非平方数分别求和。虽然这里 $a_n$ 都是正数，如果有convergence也有absolute convergence，但有点预设了你要证明的东西的感觉。
    @The Grand Inquisitor 我觉得没有问题哇，只要找到一个反例即可，如果这个反例确实满足原序列diverge 新序列converge 那不就可以了？不过为什么要提absolute convergence？
    @Cindy Zeng 我担心的是如果原序列不是absolute convergent的话，rearrangement不一定等于原序列的和。We are too young to know too much
    @The Grand Inquisitor I don’t think this is a rearrangement though. Its order is preserved, it’s just that you group them when you do addition. Is it? @The Grand Inquisitor
    @The Grand Inquisitor 天，老师最后讲rearrangement 的那个proof的时候，最后蓝笔才是正确版本对吗？相当于之前的黑笔版本就不要了？
  - Hw3 T11 (b): Back to index
    $2r_n-2\sqrt{r_nr_{n+1}}=r_n\colorbox{greenyellow}{$-r_{n+1}+r_{n+1}$}+r_n-2\sqrt{r_nr_{n+1}}=a_n+r_{n+1}+r_n-2\sqrt{r_nr_{n+1}}>a_n$ . Divide both sides by $\sqrt{r_n}$ we have the desired inequality. Since $\sum_{k=1}^n(\sqrt{r_k}-\sqrt{r_{k+1}})=\sqrt{r_1}-\sqrt{r_{n+1}}$ (telescoping), and $\lim r_n=l,\sum_{k=1}^n(\sqrt{r_k}-\sqrt{r_{k+1}})$ converges. By comparison test, … converges.

3.5 Limiting Comparison Test: Suppose $a_n,b_n \geq 0, \forall n \geq$ some $n_0$ , if $\lim_{n\to \infty} \frac{a_n}{b_n}=l$
- 辅车相依，唇亡齿寒 $0<l<\infty$ : $\sum a_n$ and $\sum b_n$ converge/diverge at the same time
- $l=0$ : convergence of $\sum b_n$ implies convergence of $\sum a_n$ and divergence of $\sum a_n$ implies divergence of $\sum b_n$
- $l=\infty$ : convergence of $\sum a_n$ implies convergence of $\sum b_n$ and divergence of $\sum b_n$ implies the divergence of $\sum a_n$
- Proof of (1) 白矛纯束:
  $\forall \epsilon>0, \exist N, s.t. \forall n \geq N: l-\epsilon<\frac{a_n}{b_n}<l+\epsilon$
  Take $\epsilon=\frac{l}{2}$ , then we have $\frac{l}{2}<\frac{a_n}{b_n}<\frac{3l}{2}, \forall n \geq N$
  Thus $\frac{l}{2}b_n<a_n<\frac{3l}{2}b_n, \forall n \geq N$ . Then by Comparison Test, if $\sum a_n$ converges, then so does $\sum\frac{l}{2}b_n$ , and so does $\sum b_n$ . Otherwise, if $\sum a_n$ diverges, then so does $\sum \frac{3l}{2} b_n$ , and so does $\sum b_n$
- Proof of (2):
  $\because \frac{a_n}{b_n}\rarr0\therefore\forall \epsilon>0,\exists N~s.t.~if~n\ge N,\frac{a_n}{b_n}<\epsilon\Rarr a_n<\epsilon b_n.$ Take $\epsilon=1\Rarr0\le a_n\le b_n\Rarr(2)$ follows from 3.4 Comparison Test.
- Examples
  - Hw3 T10 (a): Suppose $a_n>0$ , $S_n=a_1+...+a_n$ and $\sum a_n$ diverges. Prove that $\sum \frac{a_n}{1+a_n}$ diverges. Back to index
    Tian:
    Proof by contradiction: Suppose $\sum \frac{a_n}{1+a_n}$ converges, then by divergence test, $\lim \frac{a_n}{1+a_n}=0$ . Notice that $\frac{a_n}{1+a_n}=1-\frac{1}{1+a_n}$ , hence $\lim \frac{a_n}{1+a_n}=0$ implies that $\lim \frac{1}{1+a_n}=1$ .
    Take the ratio between $a_n$ and $\frac{a_n}{1+a_n}$ : $\frac{\frac{a_n}{1+a_n}}{a_n}=\frac{}{}\frac1{1+a_n}{}$ .
    Therefore, $\lim \frac{\frac{a_n}{1+a_n}}{a_n}=\lim \frac{1}{1+a_n}=1$ is a constant, $\sum \frac{a_n}{1+a_n}$ and $\sum a_n$ should diverge simultaneously, contradiction.
    Cind:
    Proof by contradiction:
    Suppose $\sum\frac{a_n}{1+a_n}$ conv $\xRightarrow{fact~3.1}\lim\frac{a_n}{1+a_n}=0\overset{\text{isolate an}}{=}\lim\frac{1+a_n}{1+a_n}-\frac{1}{1+a_n}\Rarr\lim\frac{1}{1+a_n}=1\Rarr\lim1+a_n=1\Rarr\lim a_n=0.$
    $\because a_n\overset{a_n>0}{\ge}\frac{a_n}{1+a_n}\therefore$ Comparison test (fact 3.4) doesn’t work.
    Use limiting comparison test (fact 3.5) $\lim\frac{a_n}{a_n/(1+a_n)}=\lim(1+a_n)=1\in(0,\infty).$ So $\sum a_n$ must converge with $\sum\frac{a_n}{1+a_n}$ , which contradicts with the setup.

3.6 Suppose $a_n \geq 0, \forall n\geq 1$ . Then $\sum a_n$ converges iff $\{s_n\}$ is bounded
- @The Grand Inquisitor 天，为啥我突然觉得这里有点问题， $s_n=(-1)^n$ 咋办。我的意思是如果这条fact成立那不就意味着 sn convergent $\Lrarr$ sn bounded, which is not true.?
- 这里的前提是 $a_n\geq0$ , 所以 $s_n$ 是单增的，单增的话convergence和boundedness就是可以互推的 @Cindy Zeng 哦对，我傻了 @The Grand Inquisitor

3.7 Integral Test 平行宇宙: Suppose $a_n \geq 0$ , $\{a_n\}$ is decreasing, and there exists a continuous and decreasing function $f(x)$ defined on $[1,\infty)$ s.t. $a_n=f(n)$ . Then $\sum a_n \& \int_{1}^{\infty} f(x)dx$ converge or diverge at the same time.
- Proof of Case 1: $\int_{1}^{\infty} f(x)dx$ converges.
  Since $a_n \geq 0$ , $s_n$ is increasing, to prove $\sum a_n$ converges, we only need to show $\{s_n\}$ is bounded.
  $s_n=\sum_{k=1}^{n}a_n\leq a_1+\int_{1}^{2} f(x) dx+...+\int_{n-1}^n f(x)dx=a_1+\int_{1}^{n} f(x)dx$
  Since $\int_{1}^{\infty}f(x)dx=\lim_{k \to \infty} \int_{1}^{k} f(x)dx$ exists & finite, we have $\int_{1}^{n} f(x)dx$ is bounded(Convergence implies boundedness). Therefore, $s_n$ is bounded.
- Proof of Case 2: $\int_{1}^{\infty} f(x)dx$ diverges
  Similarly, to prove $\sum a_n$ diverges, we only need to show $\{s_n\}$ is unbounded.
  $s_n =\sum_{k=1}^{n} a_n \geq \int_{1}^2 f(x)dx+...+\int_{n}^{n+1} f(x)dx= \int_{1}^{n+1} f(x)dx$
  The LHS approaches infinity as n goes to infinity, hence $\{s_n\}$ is unbounded.
- Example 9: p-series $\sum_{n=1}^\infty\frac1{n^p}$ , converges if $p>1,$ diverges if $0<p\le 1$ (or, $p\le 0$ ). Back to index
  Let $f(x)=\frac1{x^p}$ , which decreases on $[1,\infin)$ , and $f(n)=\frac1{n^p}=a_n.$
  $\int_1^\infty\frac1{x^p}dx=\int_1^\infty x^{-p}dx=\begin{cases} \frac{x^{-p+1}}{-p+1}\vert^\infty_1 & p\ne 1 \\ lnx\vert^\infty_1 & p=1\end{cases}\Rarr\int_1^\infty\frac1{x^p}dx=\begin{cases} \infty & p<1\\ \frac1{p-1} & p>1\\ \infty & p=1\end{cases}$
  $\Rarr$ By Integral Test, $\sum_{n=1}^\infty\frac1{n^p}$ converges if $p>1,$ diverges if $p\le 1$ .

3.8 Root Test: Let $\alpha=\limsup_{n\to \infty} |a_n|^{\frac{1}{n}}$
- If $\alpha <1$ , then $\sum |a_n|$ converges
  Take $\beta \in (\alpha,1)$ , Then $\exist N, s.t. |a_n|^{\frac{1}{n}}< \beta, \forall n \geq N$
  $\Rightarrow |a_n| < \beta ^n, \forall n \geq N$ .
  Since $\sum \beta^n$ as a geometric series converges, then by comparison test, $\sum |a_n|$ converges.
- If $\alpha >1$ , then $\sum |a_n|$ diverges
  Since $\limsup_{n \to \infty} |a_n|^{\frac{1}{n}}= \alpha$ , then there exists a subsequence $\{ |a_{n_k}|^{\frac{1}{n_k}}\}|_{k=1}^{\infty}$ such that $\lim_{k \to \infty} |a_{n_k}|^{\frac{1}{n_k}}=\alpha$ (F2.3.2). Consequently, take $\epsilon=\frac{\alpha-1}{2}$ (so small that $\alpha-\epsilon>1$ ), there exists $K$ such that
  $\alpha-\epsilon < |a_{n_k}|^{\frac{1}{n_k}}<a+\epsilon$
  as long as $k \geq K$ . Hence $|a_{n_k}|> (\alpha-\epsilon)^{n_k}, \forall k \geq K$ .
  But $\alpha-\epsilon>1$ , hence $\lim_{k \to \infty} (\alpha-\epsilon)^{n_k} =\infty\Rarr|a_{n_k}|\rarr\infty\Rarr\lim |a_n|\ne 0\overset{\text{div test}}{\Rarr}\sum |a_n|$ diverges.
- If $\alpha=1$ , inconclusive
  Counterexample: $\sum\frac1n$ diverges, $\sum \frac1{n^2}$ converges. Yet both of their $\alpha=\varlimsup_{n\to \infty} |a_n|^{\frac{1}{n}}=1$ .
- Example (Hw4 T1) Back to index
  洛必达+root test.
- Example (Hw4 T2) Back to index
  3个root test, 1个ratio test（见到阶乘时的条件反射）

3.9 Ratio Test $|\frac{a_{n+1}}{a_n}|$ 
- 3.9.1 If $\limsup_{n \to \infty} |\frac{a_{n+1}}{a_n}|<1$ , then $\sum |a_n|$ converges.
  We can find $\alpha<1$ and $N$ so that $|\frac{a_{n+1}}{a_n}|<\alpha$ , $\forall n \geq N$
  $\Rightarrow |a_{n+1}|<\alpha |a_n|$
  Take $n=N$ , $|a_{N+1}|<\alpha |a_N|$
  Take $n=N+1$ , $|a_{N+2}|<\alpha|a_{N+1}|< \alpha^2 |a_{N}|$
  By induction, we have $|a_{N+p}|< \alpha |a_{N+p-1}|< \alpha^p |a_{N}|$
  Then $\sum_{p=1}^{\infty}|a_{N+p}|< (\sum_{p=1}^{\infty} \alpha^p)|a_N|< \infty$ (infinite geometric series, which is finite, times a finite number).
  Thus $\sum_{n=N+1}^{\infty} |a_n|$ converges (by MCT), which implies $\sum |a_n|$ converges (by Fact 3.1).
- 3.9.2 If $\exist\, n_0,s.t. \forall n \geq n_0,|\frac{a_{n+1}}{a_n}| \geq 1,$ then $\sum a_n$ diverges.
  This means $|a_n|$ is increasing. By MCT, $\lim |a_n|=l$ exists (may be $\infty$ )
  But $|a_1|>1$ , then $\lim |a_n|=l \neq 0$ . By divergence test, $\sum a_n$ diverges
- 3.9.3 If $\liminf |\frac{a_{n+1}}{a_n}| >1$ , then $\sum a_n$ diverges.
  Follows from 3.9.2, by F2.3.1.

Theorem: Root Test is more general than Ratio Test
- $\liminf |\frac{a_{n+1}}{a_n}| \leq \liminf |a_n|^{\frac{1}{n}} \leq \limsup |a_n|^{\frac{1}{n}} \leq \limsup |\frac{a_{n+1}}{a_n}|$
- Prove $\liminf |\frac{a_{n+1}}{a_n}|\leq \liminf |a_n|^{\frac{1}{n}}$
  Let $\beta=\liminf|\frac{a_{n+1}}{a_n}| \geq 0$
  If $\beta=0$ , then $\liminf |\frac{a_{n+1}}{a_n}|=0 \leq \liminf |a_n|^{\frac{1}{n}}$ , done.
  If $0<\beta<\infty$ , then take $\gamma\in (0,\beta)$ , then there exists $N$ ,such that $\forall n \geq N$ ,
  $|\frac{a_{n+1}}{a_n}| >\gamma \Rightarrow |a_{n+1}|\geq \gamma |a_n| \Rightarrow|a_{N+p}| \geq \gamma^{p} |a_N|$
  Hence $|a_{k} |\geq \gamma^{k-N}|a_N|, \,\,\forall k \geq N+1$
  Thus $|a_k|^{\frac{1}{k}} > \gamma^{\frac{k-N}{k}}a_{N}^{\frac{1}{k}}$ . Take liminf on both sides (RHS has limits)
  $\Rightarrow \liminf |a_k|^{\frac{1}{k}} \geq \gamma$
  Since this fixed $\gamma$ is an arbitrary element from $(0,\beta)$ , we have
  $\liminf |a_k|^{\frac{1}{k}} \geq \sup\{(0,\beta)\}=\beta$
- Prove $\limsup|a_n|^{\frac{1}{n}} \leq \limsup |\frac{a_{n+1}}{a_n}|$
  Let $\alpha=\limsup |\frac{a_{n+1}}{a_n}|$ .
  If $\alpha=\infty,$ then $\limsup |a_n|^{\frac{1}{n}} \leq \infty=\limsup |\frac{a_{n+1}}{a_n}|$ .
  If $\alpha< \infty$ , then take $\gamma> \alpha$ , then there exists $N$ , such that $\forall n \geq N$ ,
  $|\frac{a_{n+1}}{a_n}|<\gamma \Rightarrow |a_{n+1}|<\gamma |a_n| \Rightarrow |a_{N+p} |<\gamma^{p} |a_{N}|$
  Thus $\forall n \geq N, |a_{n}|^{\frac{1}{n}}<\gamma^{\frac{n-N}{n}}|a_N|^{\frac{1}{n}}$ . Take limsup on both sides,
  $\limsup |a_n|^{\frac{1}{n}} \leq \gamma$
  Since this fixed $\gamma$ is an arbitrary element in $(\alpha, \infty)$ , we have
  $\limsup |a_n|^{\frac{1}{n}} \leq \inf{(\alpha,\infty)} =\alpha$
- Corollary, if $\lim |\frac{a_{n+1}}{a_n}|$ exists, then $\lim |a_n|^{\frac{1}{n}}$ also exists and equals $\lim |\frac{a_{n+1}}{a_n}|$
  Proof: Squeezing Principle
- Example 10: Try root test and ratio test on series: $\frac12+\frac13+\frac1{2^2}+\frac1{3^2}+...$ Back to index
  Root test:
  $a_{2k-1}=\frac1{2^k},\forall k\ge0,a_{2k}=\frac1{3^k},\forall k\ge 1.$ Want to know $\varlimsup |a_n|^\frac1n.$
  Let $\varlimsup |a_n|^\frac1n=\sup E.~|a_n|^\frac1n=\begin{cases} \frac1{2^\frac{k}{2k-1}}, & n=2k-1 & \rarr\frac1{\sqrt2}\\ \frac1{3^\frac{k}{2k}},& n=2k &=\frac1{\sqrt3} \end{cases}\Rarr E={\frac1{\sqrt2},\frac1{\sqrt3}},\sup E=\frac1{\sqrt2}<1.$
  By root test the series converges absolutely.
  Ratio test:
  Want to know $\varlimsup \frac{a_{n+1}}{a_n}.$
  $\frac{a_{n+1}}{a_n}=\begin{cases} (\frac23)^k, & n=2k-1\\ (\frac32)^k\cdot\frac12, & n=2k \end{cases}\Rarr \varlimsup \frac{a_{n+1}}{a_n}=\infty,\varliminf \frac{a_{n+1}}{a_n}=0$
  Ratio test fails.
  Root test prevails.

Power Series: $\sum a_nx^n$
- Central Question: For what $x$ does the power series converge or diverge?
- Theorem: Let $\alpha=\limsup_{n \to \infty} |a_n|^{\frac{1}{n}}$ , Let $R=\frac{1}{\alpha}$ .
  - For $x \in (-R,R)$ , $\sum a_n x^n$ converges absolutely.
  - For $|x| > R$ , $\sum a_nx^n$ diverges.
  - Remark: The theorem does not apply to points where $x=\pm R$
    @The Grand Inquisitor 天，可以把你这里老师讲does not apply 的时候举的例子那部分笔记拍给我一下吗，我这里记的interval of convergence 是[-1,1), 但看过程应该是（-1，1）啊？我的搞错啦！确实应该是[-1,1)吧
    $\sum \frac{(-1)^n}{n}$ 是converge的，由alternating test
    嗯嗯对！谢谢小天。我刚刚才看到alternating series test. 才反应过来。
  - Proof
    Let $c_n=a_nx^{n}$ , then $\sum a_nx^n=\sum c_n$ .
    Use the Root Test, $\limsup_{n\to \infty} |c_n|^{\frac{1}{n}}=\limsup_{n \to \infty}|a_n|^{\frac{1}{n}}|x|= \alpha |x|$
    By Root Test, if $\alpha|x|<1$ , then $\sum c_n$ converges absolutely, this requires $|x|<\frac{1}{\alpha}$
    Similarly, if $\alpha|x| >1$ , then $\sum c_n$ diverges, this requires $|x|> \frac{1}{\alpha}$
  - Example 11
    See here.

$\sum a_nb_n$
- Theorem: the partial sums $\{A_n\}$ of $\sum_{n=0}^{\infty} a_n$ are bdd. $\{b_n\}_{n=0}^{\infty}$ is decreasing and approaches zero. Then $\sum a_nb_n$ converges.
  - Preparation: Summation by parts
    Let $A_n=\sum_{k=0}^{n}a_k$ . The for $0\leq p\leq q$ , we have
    $\sum_{n=p}^{q} a_nb_n=A_qb_q-A_{p-1}b_p-\sum_{n=p}^{q-1} A_n(b_{n+1}-b_n)$
    Proof:
    $a_n=A_n-A_{n-1,}$ 之后进行拆分尽力往desired样子靠就行了。
  - Proof
    $\forall n\geq m\geq 1$ , we have
    $|\sum_{k=m}^{n} a_kb_k|=|A_nb_n-A_{m-1}b_m-\sum_{k=m}^{n-1} A_k(b_{k+1}-b_k)|$
    $\leq |A_n|b_n+|A_{m-1}|b_m+|\sum_{k=m}^{n-1} A_k(b_{k+1}-b_k)|$
    (Remark: no absolute value before $b_n$ because $b_n$ is positive)
    Since $\{A_n\}$ is bounded, we can find a $M$ such that $|A_n|<M$ . Then
    $\leq M(b_n+b_m)+M\sum_{k=m}^{n-1}(b_{k+1}-b_k)$
    $\sum_{k=m}^{n-1}(b_{k+1}-b_k)=b_{m+1}-b_m+b_{m+2}-b_{m+1}+...+b_{n}-b_{n-1}=b_n-b_m$ . (Remark: This is called the telescoping series, as all the barriers between the beginning and the ending get killed) . Then we have
    $\leq M(b_n+b_m)+M(b_n-b_m)=2Mb_n$
    Again, no absolute value in the second term because $\{b_n\}$ is decreasing.
    Since $\lim b_n=0$ , then $\forall \epsilon>0, \exist N,s.t.\,\,\forall n \geq N,b_n<\epsilon$ , thus
    $\forall \epsilon>0, \exist N, s.t. \forall n\geq m\geq N, |\sum_{k=m}^{n} a_kb_k|\leq 2Mb_n \leq 2M\epsilon$
    Hence $\sum a_nb_n$ converges.
  - 换汤不换药：
    Hw4 T3: If $\sum a_n$ converges, and if $\{b_n\}$ is monotonic and bounded, prove that $\sum a_nb_n$ converges. Back to index
    Rudin
    We shall show that the partial sums of this series form a Cauchy sequence, i.e., given $\varepsilon>0$ there exists $N$ such that $|\sum_{k=m}^n a_kb_k|<\varepsilon$ if $n\geq m\geq N$ .
    By summation by parts, we have
    $\sum_{k=m}^n a_kb_k=A_nb_n-A_{m-1}b_m+\sum_{k=m}^{n-1} A_n(b_{k+1}-b_{k})$
    Divide and conquer: Denote $\lim A_n=A, \lim b_n=b,$ and $M$ be a bound of $\{A_n\}$ .
    $|\sum_{k=m}^n a_kb_k|=|A_nb_n\colorbox{greenyellow}{$-Ab+Ab$}-A_{m-1}b_m+\sum_{k=m}^{n-1} A_k(b_{k+1}-b_k)|$ $\\|\sum_{k=m}^n a_kb_k|=|A_nb_n-Ab+Ab-A_{m-1}b_m+\sum_{k=m}^{n-1} A_k(b_{k+1}-b_k)|$
    $\leq |A_nb_n-Ab|+|A_{m-1}b_m-Ab|+\sum_{k=m}^{n-1} |A_k||b_{k+1}-b_k|$
    $\leq |A_nb_n-Ab|+|A_{m-1}b_m-Ab|+M\sum_{k=m}^{n-1}|b_{k+1}-b_k|$
    Since $\lim A_nb_n=Ab$ , we can find $n,m$ big enough such that
    $|A_nb_n-Ab|<\varepsilon, |A_{m-1}b_m-Ab|<\varepsilon$
    We only need to worry the last term. Given the sequence $\{b_n\}$ is monotonic,
    $\sum_{k=m}^{n-1} |b_{k+1}-b_k|=|\sum_{k=m}^{n-1} b_{k+1}-b_k|=|b_n-b_m|$
    where $|b_n-b_m|<\varepsilon$ by Cauchyness. Hence
    $|\sum_{k=m}^{n} a_kb_k|\leq |A_nb_n-Ab|+|A_{m-1}b_m-Ab|+M|b_n-b_m|\leq (2+M)\epsilon$
- Alternating Series Test: Suppose $\{b_n\}_{n=0}^{\infty}$ is decreasing and approaches zero. Then $\sum (-1)^n b_n \&\sum_{-1}^{n+1} b_n$ converges. @The Grand Inquisitor 你这里写错了，是 $\sum (-1)^n b_n \&\sum(-1)^{n+1} b_n$ 吧

Additions & Multiplication of Series
- Addition
  Suppose $\sum_{n=n_0}^\infty a_n~\&~\sum_{n=n_0}^\infty b_n$ both converges, then
  (1) $\sum_{n=n_0}^\infty (a_n\pm b_n)$ also converges.
  (2) $\sum ca_n=c\sum a_n$ also converges.
- Multiplication
  - Cauchy product: Give $\sum_{n=0}^{\infty} a_n$ and $\sum_{n=0}^{\infty} b_n$ . Define Cauchy product of these two series by $\sum_{n=0}^{\infty} c_n$ , where $c_n=\sum_{k=0}^{n} a_kb_{n-k},\forall n \geq 0$ .
  - Theorem: If $\sum_{n=0}^{\infty} a_n$ converges absolutely, $\sum_{n=0}^{\infty} b_n$ converges, then Cauchy product $\sum_{n=0}^{\infty} c_n$ converges and equals $AB$ , where $A=\sum_{n=0}^{\infty} a_n, B=\sum_{n=0}^{\infty} b_n$
    $C_n=c_0+c_1+...c_n$
    $=(a_0b_0)+(a_1b_0+a_0b_1)+...+(a_0b_n+a_1b_{n-1}+...+a_nb_0)$
    $=a_0B_n+a_1B_{n-1}+...+a_n B_0$
    $=a_0(B_n-B)+a_1(B_{n-1}-B)+...+a_n(B_0-B)+B(a_0+...+a_n)$
    $=[a_0(B_n-B)+a_1(B_{n-1}-B)+...+a_n(B_0-B)]+BA_n$
    The second term approaches to our desired result $AB$ ; part of the first term vanishes to zero as $|B_n-B|$ approaches to zero, but terms like $|B_0-B|$ is not tiny.
    Now we proceed with the formal language,
    Since $\lim B_n=B$ , then $\forall \epsilon>0, \exists N,s.t.\,\,\forall n\geq N, |B_n-B|<\epsilon$ , thus
    $|a_0(B_n-B)+a_1(B_{n-1}-B)+...+a_n(B_0-B)|$
    $\leq |a_0||B_n-B|+...+|a_{n-N}||B_N-B|+...+|a_n||B_0-B|$
    In the first $n-N$ terms, $|B_{k}-B|<\epsilon$ ; in the remaining $N$ terms, $|B_k-B| \geq \epsilon$
    $\leq \epsilon \sum_{k=0}^{n-N} |a_k|+|a_{n-N}||B_{N-1}-B|+...+|a_n||B_0-B|$
    $\leq \epsilon \sum_{k=0}^{\infty} |a_k|+|a_{n-N}||B_{N-1}-B|+...+|a_n||B_0-B|$
    Road 1:
    Since $\{B_n\}$ converges, $\{B_n\}$ is bounded, hence $\exist M,s.t. \,\, B_n \leq M, \forall n$ , we have
    $\leq \epsilon \sum_{k=0}^{\infty} |a_k|+(M+|B|)\sum_{k=0}^{n-N} |a_k|$
    Take $\epsilon'=\frac{1}{N(\epsilon)(M+|B|)}\epsilon$ , then since $\lim |a_n|=0$ , there exists $N'$ , such that $|a_n|<\frac{1}{N(\epsilon)(M+|B|)}\epsilon, \forall n\geq N'$ . Hence we get
    $\leq \epsilon \sum_{k=0}^{\infty} |a_k|+(M+|B|) N(\epsilon) \frac{1}{N(\epsilon)(M+|B|)}\epsilon=(1+\sum_{k=0}^{\infty} |a_k|) \epsilon$
    Road 2:
    take $\varlimsup$ on both sides.
    $\varlimsup RHS=\epsilon\sum_{k=1}^\infty|a_k|+0+0+...(F3.1)$ $\Rarr0\le LHS\le\epsilon\sum_{k=1}^\infty|a_k|,\forall \epsilon\Rarr LHS=0.\blacksquare$

The Magic Number e

Example 4: Does $\lim_{n\to \infty} (1+\frac{1}{n})^n$ exist? (The definition of $e$ ; proved by M.C.T.) Back to index
We prove the existence of limits by M.C.T.
Firstly, we show $\{(1+\frac{1}{n})^n\}_{n=1}^{\infty}$ is monotonically increasing.
Recall the binomial theorem: $(a+b)^n=\sum_{k=0}^n C_{n}^{k} a^{n-k}b^k$ where $C_n^k=\binom{n}{k}=\frac{n!}{k!(n-k)!}$
$(1+\frac{1}{n})^n=1+\frac{n}{1}\frac{1}{n}+\frac{n(n-1)}{2!}(\frac{1}{n})^2+\frac{n(n-1)(n-2)}{3!}(\frac{1}{n})^3+...+\frac{n!}{n!}(\frac{1}{n})^n$
$\overset{\text{turn coefficients in to the form of }1/n!}{=}1+1+\frac{1}{2!}(1-\frac{1}{n})+\frac{1}{3!}(1-\frac{1}{n})(1-\frac{2}{n})+...+\frac{1}{n!}(1-\frac{1}{n})(1-\frac{2}{n})...(1-\frac{n-1}{n})$
$\overset{\text{turn every denom n into n+1}}{<} 1+1+\frac{1}{2!}(1-\frac{1}{n+1})+\frac{1}{3!}(1-\frac{1}{n+1})(1-\frac{2}{n+1})+...+\frac{1}{n!}(1-\frac{1}{n+1})...(1-\frac{n-1}{n+1})<\text{this bunch} + \frac{1}{(n+1)!}(1-\frac{1}{n+1})...(1-\frac{n}{n+1})$
$=(1+\frac{1}{n+1})^{n+1}$
Secondly, we show that $\{(1+\frac{1}{n})^n\}_{n=1}^{\infty}$ is bounded from above
$(1+\frac{1}{n})^n=1+1+\frac{1}{2!}(1-\frac{1}{n})+\frac{1}{3!}(1-\frac{1}{n})(1-\frac{2}{n})+...+\frac{1}{n!}(1-\frac{1}{n})...(1-\frac{n-1}{n})$
$\leq 1+ (1+\frac{1}{2!}+\frac{1}{3!}+...+\frac{1}{n!})$
$\leq 1+(1+\frac{1}{2}+\frac{1}{2^2}+...+\frac{1}{2^{n-1}})$
$=1+\frac{1-\frac{1}{2^n}}{1-\frac{1}{2}} <3$
Now by MCT, the limit exists as a finite number, which is $e$ .

Example 7: Prove $\lim_{n\to \infty} (1+1+\frac{1}{2!}+\frac{1}{3!}+...+\frac{1}{n!})=e$ . (proved by limsup and liminf) Back to index
We do not even know whether $\lim_{n\to \infty} (1+1+\frac{1}{2!}+\frac{1}{3!}+...+\frac{1}{n!})$ exists or not, hence we can only use limsup and liminf to manipulate the inequalities.
Ideally, we want to show
$\limsup (1+1+\frac{1}{2!}+...+\frac{1}{n!}) \leq e\leq \liminf (1+1+\frac{1}{2!}+...+\frac{1}{n!})$
Firstly, we show that $e \leq \liminf(1+1+\frac{1}{2!}+...+\frac{1}{n!})$
$(1+\frac{1}{n})^n=1+1+\frac{1}{2!}(1-\frac{1}{n})+\frac{1}{3!}(1-\frac{1}{n})(1-\frac{2}{n})+...+\frac{1}{n!}(1-\frac{1}{n})...(1-\frac{n-1}{n})$
$< 1+1+\frac{1}{2!}+\frac{1}{3!}+...+\frac{1}{n!}$
Take liminf on both sides of the inequality
$\liminf (1+\frac{1}{n})^n \leq \liminf (1+1+\frac{1}{2!}+\frac{1}{3!}+...+\frac{1}{n!})$
But since $\lim (1+\frac{1}{n})^n=e$ , we have $\limsup (1+\frac{1}{n})^n=\liminf (1+\frac{1}{n})^ n=e$
Hence we showed that $e\leq\liminf (1+1+\frac{1}{2!}+...+\frac{1}{n!})$
Secondly, we show that $\limsup (1+1+\frac{1}{2!}+...+\frac{1}{n!}) \leq e$
If we can show that $e \geq \sup \{1+1+\frac{1}{2!}+...+\frac{1}{n!}\}_{n=1}^{\infty}$ , since $\sup \geq \limsup$ , then we are done. To accomplish this goal, we only need to show that
$e \geq 1+1+\frac{1}{2!}+..+\frac{1}{m!},\forall m$
Fix an arbitrary $m$ , for any $n \geq m$ , we have
$(1+\frac{1}{n})^{n} \geq 1+1+\frac{1}{2!} (1-\frac{1}{n})+\frac{1}{3!}(1-\frac{1}{n})(1-\frac{2}{n})+...+\frac{1}{m!}(1-\frac{1}{n})...(1-\frac{m-1}{n})$
Essentially, we are dumping the elements after $m$ . Since $m$ is a fixed finite number, we can take limits on both sides of the inequality
$\lim_{n \to \infty} (1+\frac{1}{n})^{n} \geq \lim_{n \to \infty} 1+1+\frac{1}{2!}(1-\frac{1}{n})+...+\frac{1}{m!}(1-\frac{1}{n})...(1-\frac{m-1}{n})$
$\Rightarrow e \geq 1+1+\frac{1}{2!}+\frac{1}{3!}+...+\frac{1}{m!}$
Since this fixed $m$ is arbitrary, we actually have $e \geq 1+1+\frac{1}{2!}+...+\frac{1}{m!}, \forall m$ .
$e\geq \sup \{1+1+\frac{1}{2!}+...+\frac{1}{n}\}_{n=1}^{\infty} \geq \limsup 1+1+\frac{1}{2!}+...+\frac{1}{n!}$

Note: Taylor expansion: $e^x=1+x+\frac{x^2}{2!}+\frac{x^3}{3!}+...$ so $e=1+1+\frac{1}{2!}+\frac{1}{3!}+...$

Example 11: What is the interval of convergence for $\sum_{n=0}^\infty 1+x+\frac{x^2}{2!}+\frac{x^3}{3!}+...(0!=1)$ . This is the Taylor series of $e^x$ . Back to index
Solution:
First try root test: $a_n=\frac1{n!},\varlimsup a_n^{\frac1n}=\varlimsup \frac 1{(n!)^\frac1n}$ , ugly, try ratio test (will come to learn that $n!\overset{conditional~reflex}{\rarr}\text{ratio test.}$
Try ratio test: let $c_n=\frac{x^n}{n!},\lim{|c_{n+1}|}{|c_n|}=\lim\frac{|x|}{n+1}\overset{|x|\text{ is arbitrarily fixed}}{=}0<1$ . By ratio test, $\sum c_n$ converges absolutely, $\forall x\in\R$ . So interval of convergence: $(-\infin,\infin)$ .

Hw3 T6: Prove that $\{1+\frac{1}{2}+...+\frac{1}{n}-\ln(n+1)\}$ is convergent. (The limit is called Euler’s constant, denoted by $\gamma$ ). Back to index
- Hints
  - Use $\frac{1}{k+1}<\int_{k}^{k+1} \frac{1}{x} dx<\frac{1}{k}$ , $\forall k \geq1$ . 第一个不等号证bdd+之后放缩, 第二个证mono。Reminds me of proof of integral test, any strictly decreasing $f(x)$ would satisfy I guess.
  - Show $1+\frac{1}{2}+...+\frac{1}{n}-\ln(n+1)$ is monotone and bounded.
- Proof
  Denote $S_n=1+\frac{1}{2}+...+\frac{1}{n}-\ln(n+1)$
  Then $S_{n}-S_{n-1}=\frac{1}{n}+\ln(n)-\ln(n+1)$
  Since $\int_{n}^{n+1} \frac{1}{x}dx=\ln(n+1)-\ln(n)<\frac{1}{n}$ , we have $S_n-S_{n-1}>0$ .
  To prove $S_n$ is bounded, Notice that $\ln (n+1)\geq \frac{1}{n+1}+\ln(n)$ , hence
  $S_n<1+\frac{1}{2}+...+\frac{1}{n}-\ln(n)-\frac{1}{n+1}$
  We want to make the RHS bigger, use $\frac{1}{n+1}<\int_{n}^{n+1} \frac{1}{x} dx$ , thus
  $S_n<1+\int_{1}^2 \frac{1}{x} dx+\int_{2}^3 \frac{1}{x}dx+...+\int_{n-1}^n \frac{1}{x}dx -\ln(n)-\frac{1}{n+1}$
  $<1+\int_1^n \frac{1}{x} dx-\ln(n)-\frac{1}{n+1}=1+\ln(n)-\ln(n)-\frac{1}{n+1}<1$
  (Remark1: we do not enlarge the first term because $\ln(0)$ is not defined or is $-\infty$ )
  (Remark2: This trick shares the same spirit as the integral test)

4. Basic Point-Set Topology

Topology, no trick, all conceptual problems

Definitions
- Metric space
Let $X$ be a metric space. All points and sets mentioned below are understood to be elements and subsets of $X$ .
- Neighborhood
  - Let $p\in X$ , a neighborhood of $p$ is a ball with radius $r$ $N_r(p)=\{q\in X|d(p,q)<r\}$ . Later on, we can use “ $q$ is within a small neighborhood of $p$ ’’ to replace “ $q$ is very close to $p$ .”
- Limit point
  - A point $p\in X$ is said to be a limit point of $E$ if $\forall r>0,N_r(p)$ contains a point $q\neq p$ with $q\in E$ .
- Closed set
  - Let $E'$ be the set of all limit points of $E$ . Then $E$ is said to be closed if $E'\subset E$
- Interior point
  - A point $p$ is said to be an interior point of $E$ if $\exists$ nbhd $N_r(p)\subset E$ .
- Open set
  Let $\dot{E}$ be the set of all interior point of $E$ . $E$ is said to be open, if every point in $E$ is an interior point of $E$ , i.e. $\dot{E}=E$ .
- Bounded set
  $E$ is bounded if there is a real number $M$ and a point $q\in X$ such that $d(p,q)<M$ for all $p\in E$ .
- Clousure
  - The closure of $E$ is defined to be $\bar{E}=E'\cup E$
- Dense set
  E is dense in $X$ if every point of $X$ is a limit point of $E$ , or a point of $E$ (or both), i.e. $X\subset \bar{E}$ .
- Finite, Countable, and Uncountable Sets
  - A set $A$ is said to be finite if either $A=\empty$ or $A=\{a_1,...,a_n\}$ .
  - A set $A$ is said to be countable if $A$ is infinite and all its elements can be listed as a sequence $\{a_1,a_2,...,a_n...\}$ .
    $N^{+}=\{n\}_{n=1}^{\infty}$ is countable.
    the interval $(0,1)$ is not countbale.
  - A set $A$ is said to be at most countable if $A$ is either finite or countable.
  - We say $A$ is equivalent to $B$ (denoted as $A\sim B$ ) if $\exist$ function $f:A\to B$ such that $f$ is 1-1 and onto.
  - Basic facts
    $A$ is countable $\Leftrightarrow A\sim N^{+}=\{1,2,...\}$ .
    A subset of a countable set is at most countable.
    If each $E_n$ is countable, then $\cup_{n=1}^{\infty} E_n$ is also countable.
    If each $A_n$ is countable, $n=1,...,N$ . Then $A_1\times A_2\times ...\times A_N\equiv \{(a_1,...,a_N)|a_i\in A_i\}$ is countable.
    $Q=$ set of all rational # is countbale.

Basic Facts
- Basic facts 1: Any nbhd $N_r(p)$ is open
  Just need to prove $\forall q\in N_r(p)$ , $q$ is an interior point of $N_r(p)$ .
  Use $\delta=\frac{1}{2}[r-d(p,q)]$ as radius to construct $N_{\delta}(q)$ , w.t.s. $N_{\delta}(q)\subset N_{r}(p)$ .
  $\forall s\in N_{\delta}(q)$ :
  $d(s,p)\leq d(s,q)+d(q,p)<\delta+d(p,q)=\frac{r-d(p,q)+2d(p,q)}{2}=\frac{r+d(p,q)}{2}<r$
  Hence $s\in N_{r}(p)\Rightarrow N_{\delta}(q)\subset N_r(p)$ , and $N_r(p)$ is open.
- Basic facts 2: $p$ is a limit point of $E$ iff $\forall r>0$ , $N_r(p)$ contains infinitely many points of $E$ (2); iff $\exist \{q_n\}\subset E$ such that $q_n\neq q_m$ for $n\neq m$ and $q_n\to p$ .
  - (2) $\Rightarrow$ (1): obvious
  - (1) $\Rightarrow$ (2)
    Argue by contradiction. Suppose not, then $\exist r_{bad}>0$ , such that $N_{r_{bad}}(p)$ contains only finite many point $\{e_1,...,e_n\}$ of $E$ .
    Define $r_0=\frac{\min\{d(p,e_1),...,d(p,e_2)\}}{705}$ , then $N_{r_0}(p)$ contains no points $\neq p$ and $\in E$
    Notice that the min exists due to finiteness.
  - (3) $\Rightarrow$ (1)
    Since $d(q_n,p)\to 0$ as $n\to \infty$ ,
    $\forall r>0$ , $\exist N$ such that if $n\geq N$ , we have $d(q_n,p)<r$ .
    Hence $q_N\in N_r(p)$ and $q_n\in E$ . What if $q_N=p$ ? Since $q_{N+1}\neq q_{N}$ and $q_{N+1}\in N_r(p)$ , we can let $q_{N+1}$ be the desired point.
  - (1) $\Rightarrow$ (3)
    Construct such a sequence. Since $p$ is a limit point of $E$ , then $\forall r>0, N_r(p)$ contains point $q\neq p \in E$ .
    Take $r_1=1,$ $N_1(p)$ contains point $q_1\in E, q_1\neq p$ .
    Next time, take $r_2=\frac{1}{2}d(p,q_1)$ , within a smaller neighborhood, we can find $q_2\in E, q_2\neq p, q_2\neq q_1$ .
    By induction, take $r_n=\frac{1}{2}d(p,q_{n-1})$ , and we can find $q_n$ in $N_{r_n}(p)$ such that $q_n\in E, q_n\neq p$ , $q_{n}\neq q_{m},\forall m<n$ .
    The last thing is to prove the sequence $\{q_n\}\subset E$ we construct converges to $p$ .
    By induction, we can show that $r_n<\frac{1}{2^{n-1}}$ , hence $d(q_n,p)\to 0$ , so $q_n\to p$ .
  - Weaker version, endorsed by TA: $x\in \bar{A}\Leftrightarrow \exist \{x_n\}\subset A,s.t.\,x_n\to x$ .
  - Related fact: $\forall x\in B,\exist \{x_n\}\subset A,$ s.t. $x_n\to x$ $\Leftrightarrow A$ is dense in $B$ .
- Basic facts 3: Let $E^{c}$ be the complement of $E$ which is defined by $E^{c}=X/E$ . Then $E$ is open iff $E^{c}$ is closed. (Usually, it is easier to prove openness).
  - $\Rightarrow$
    W.T.S. $(E^{c})'\subset E^c$ . Argue by contradiction. Suppose $\exist p\in (E^c)'$ and $p\in E$ . Since $E$ is open, $p$ is an interior point of $E$ , then $\exist r,s.t. N_r(p)\subset E$ . This $N_r(p)$ contains no point in $E^c$ , contradicting to $p\in (E^c)'$ .
  - $\Leftarrow$
    W.T.S. $\forall p\in E$ is an interior point of $E$ . Suppose not, then $\exist p\in E$ , such that $\forall r,$ $N_r(p)$ contains points in $E^c$ . These points can not equal $p$ as $p\in E$ , so we have $p$ is a limit point of $E^c$ . But given $E^c$ is closed, $p\in (E^c)'$ implies that $p\in E^c$ , contradicting to $p\in E$ .
- Basic facts 4 Unions and Intersections of Open(Closed) Sets:
  - Let $\{G_{\alpha}\}_{\alpha\in A}$ be a family of open sets of $X$ , then $\cup_{\alpha\in A} G_{\alpha}$ is open.
    $\forall p\in \cup_{\alpha \in A} G_{\alpha} \Rightarrow p\in$ some $G_{\alpha}$ . Since $G_{\alpha}$ is open, then $\exist$ nbhd $N_r(p)\subset G_{\alpha}\subset\cup_{\alpha\in A}G_{\alpha}$ . Thus $p$ is an interior point of $\cup_{\alpha\in A} G_{\alpha}$ .
  - Let $\{F_{\alpha}\}_{\alpha \in A}$ be a family of closed sets of $X$ , then $\cap_{\alpha \in A}F_{\alpha}$ is closed.
    Use fact 3, $(\cap_{\alpha\in A} F_{\alpha})^{c}=\cup_{\alpha \in A}F_{\alpha}^c$ . $F_{\alpha}^c$ is open by fact 3, and $\cup_{\alpha\in A} F_{\alpha}^c$ is open by fact4(i), hence $\cap_{\alpha\in A} F_a$ is closed.
  - In (1), if $A$ is finite, then $\cap_{\alpha \in A} G_{\alpha}$ is open.
    Given $A$ is finite, denote $A=\{\alpha_1,...,\alpha_n\}$ . Then $\cap_{\alpha \in A} G_{\alpha}=\cap_{i=1}^n G_{\alpha_i}$ .
    $\forall p\in \cap_{i=1}^n G_i \Rightarrow p\in G_i, i=1,...,n.$ Since $G_{\alpha_i}$ is open, then $\exist N_{r_i}(p)\subset G_{\alpha_i}$
    Now let $r=min(r_1,...r_n)$ , then $N_r(p)\subset N_{r_i}(p)\subset G_{\alpha_i},\forall i=1,...,n$ , hence $N_r(p)\subset \cap_{i=1}^n G_{\alpha_i}$ , done. Again, the existence of min requires finiteness.
  - In (2), if $A$ is finite, then $\cup_{\alpha \in A} F_{\alpha}$ is closed.
    Use the same trick in proving (2). By fact 3,
    $(\cup_{i=1}^n F_{\alpha_i})^c=\cap_{i=1}^n F_{\alpha_i}^c$ . By (3) and fact 3, $\cap_{i=1}^n F_{\alpha_i}^c$ is open and $\cup_{i=1}^n F_{\alpha_i}$ is closed.
- Basic facts 5 Closure:
  - Closure $\bar{E}$ of $E$ is closed.
    Use fact 3, w.t.s. $(\bar{E})^c$ is open.
    $\forall p\in (\bar{E})^c$ , we have $p\notin E, p\notin E'$ . Hence there exists a bad $r_{bad}$ , such that $N_{r_{bad}}(p)$ does not contain a point $q\in E$ . Hence $N_{r_{bad}}(p)\subset E^c$ .
    We want to show $N_{r_{bad}}(p)\subset (\bar{E})^c$ , but $(\bar{E})^c=(E\cup E')^c=E^c\cap (E')^c$
    So we need to show $N_{r_{bad}}(p)\subset (E')^c$ , or equivalently, $N_{r_{bad}}(p)\cap E'=\empty$ .
    Argue by contradiction, suppose $N_{r_{bad}}(p)\cap E'\neq \empty$ , then $\exist q\in N_{r_{bad}}(p)$ and $q\in E'$ . Since $q$ is a limit point of $E$ , then $\forall r$ , we have $N_r(q)$ contains a point in $E$ . We can choose $r$ so small that $N_r(q)\subset N_{r_{bad}}(p)$ (this is doable because neighborhood is open), then $N_{r_{bad}}(p)$ would contain a point in $E$ , contradiction.
  - $E=\bar{E}\Leftrightarrow E$ is closed.
    $\Rightarrow$ is obvious because of $E$ . As for $\Leftarrow:$ Since $E$ is closed, $E'\subset E$ , we havd $E\cup E'\subset E$ . But we also have $E\subset E\cup E'$ , hence $E=E\cup E'=\bar{E}$ .
  - If $F$ is closed in $X$ and $E\subset F$ , then $\bar{E}\subset F$ , i.e. $\bar{E}$ is the smallest closed set containing $E$ .
    $E\subset F\Rightarrow E'\subset F' \Rightarrow E\cup E' \subset F\cup F'\Rightarrow \bar{E}\subset \bar{F}=F$ .

Hw 5 Q1:

Hw 5 Q2: Let $A_1,A_2,...$ be subsets of a metric space: (i) If $B_n=\cup_{i=1}^n A_i$ , prove that $\bar{B}_n=\cup_{i=1}^n \bar{A}_i$ , for $n=1,2,3,...$ ; (ii) If $B=\cup_{i=1}^{\infty} A_i,$ prove that $\cup_{i=1}^{\infty} \bar{A}_i \subset \bar{B}$ .
- Key argument: If $p$ is a limit point of $B$ , then it must be the limit point of some $A_i$ . Suppose not, then for each $i$ , exists $r_i$ , such that $N_{r_i}(p)$ does not contain any point in $A_i$ . Take $r=\min\{r_1,...,r_n\}$ (minimum exists in finite many numbers), then $N_r(p)$ does not contain any point in $B$ . Similar to the prove of $\cap_{i=1}^n A_i$ is open and $\cup_{i=1}^n A_i$ is closed.

Hw 5 Q3 : Let $\dot{E}$ denote the set of all interior points of $E$ . (a) Prove that $\dot{E}$ is always open. (c) If $G\subset E$ and $G$ is open, prove that $G\subset \dot{E}$ . (d) Prove that the complement of $\dot{E}$ is the closure of the complement of $E$ . (e) Do $E$ and $\bar{E}$ always have the same interiors? (f). Do $E$ and $\dot{E}$ always have the same closure?
- (a) $Nr(x)$ , consider $p\in N_{\frac{r}{3}}(x)$ , then $N_{\frac{r}{3}}(p)\subset N_r(x)\subset E$ , hence every point of $N_{\frac{r}{3}}(x)$ is an interior point of $E$ ; (d) Two-side inclusion.

Hw 5 Q4: Let $X$ be a metric space and $E\subset X$ . The boundary of $E$ is defined to be $\{x\in X |\forall Nr(x)$ , $N_r(x)$ contains a point of $E$ and a point of $E^c\}$ . The boundary is denoted as $\partial E$ . Prove: (a) $\partial E$ is closed. (b) $\bar{E}=\dot{E}\cup \partial E$ . (c) Is it always true that a boundary point must be in $E$ .
- (a)Usually, it is easy to prove open (exist is easy than any). Consider proving $(\partial E)^c$ is open. $\forall x\in(\partial E)^c$ , $\exist N_r(x)$ , sucht that $N_r(x)\cap E=\empty$ or $N_r(x)\cap E^c=\empty$ . Consider $\forall p\in N_{\frac{r}{3}}(x)$ , then $N_{\frac{r}{3}}(p)\subset N_r(x)$ , thus $p\in (\partial E)^c$ .
- (b) Two-side inclusion

Hw 5 Q5: Let $E\subset R$ be bounded from above. Prove that $\sup E\in \bar{E}$
- Connect the definition of sup and the definition of limit points.

5. Compact Sets

Definition: A subset $K$ of a metric space $X$ is said to be compact if every open cover of $K$ contains a finite subcovers. More explicitly, the requirement is that if $\{G_{\alpha}\}$ is an open cover of $K$ , then there are finite many indices $\alpha_1,...,\alpha_n$ such that $K\subset G_{\alpha_1}\cup...\cup G_{\alpha_n}$ .
- Trick to construct an open covering: $\cup_{q\in K} N_{r_{q}}(q)$ .

Compact set is closed
- Prove $E^c$ is open. Key construction: $\forall p\in E^c, \forall q\in E$ , since $p\neq q$ , we can find a small $r_{q}$ sucht that $p\notin N_{r_q}(q)$ . But $\cup_{q\in E} N_{r_q}(q)$ is an open covering of $E$ , hence exists finite many $q_1,...,q_n$ s.t. $E\subset\cup_{i=1}^n N_{r_{q_i}}(q_i)$ . Again, minimum exists for finite many numbers, we can find a small $r=min\{d(p,q_i)-r_{q_i}\}$ such that $N_r(p)\cap N_{r_{q_i}}(q_i)=\empty$ , hence $p$ is an interior point of $E^c$ .

Compact set is bounded (Hw6 Q3)
- Key construction: $\forall x\in E,$ let $N_1(x)$ be the neighborhood of $x$ with radius $1$ . Then $\cup_{x\in E} N_1(x)$ is an open covering of $E$ .
- Useful fact, finite number has maximum.

Heine-Borel: Let $E\subset R^k$ , then $E$ is compact $\Leftrightarrow$ $E$ is closed and bounded (Hw 6 Q4)
- Induction by dimension: use Bolzano-Weierstrass in each dimension.
- Application: Rectangle in $R^n$ is compact (Hw 6 Q5)

Sequential compact $\Leftrightarrow$ Compact (Hw 6 Q1&Q2)
- A subset $K$ of metric space $X$ is said to be sequentially compact if $\forall \{p_n\}_{n=1}^{\infty}\subset K$ , $\exist$ convergent subsequence $\{p_{n_k}\}_{k=1}^{\infty}$ , s.t. $p_{n_k}\to p_{\infty}\in K$ , as $k\to \infty$ .
- Hw6 Q1: Compactness $\Rightarrow$ Sequentially Compactness
  - Case 1: The set $\{p_n\}_{n=1}^{\infty}$ is a finite set, i.e. $\exist \bar{p}\in \{p_n\}_{n=1}^{\infty}$ and $\bar{p}$ occurs infinitely many times in $\{p_{n}\}_{n=1}^{\infty}$ , we can take $\{p_{n_k}\}_{k=1}^{\infty}=\{\bar{p}\}$ , then $p_{n_k}\to \bar{p}\in K$ .
  - Case 2: The set $\{p_n\}_{n=1}^{\infty}$ is infinite (i.e. it contains infinitly many distinctive points). Let $E=\{p_{n}\}_{n=1}^{\infty}$ .
    Subcase1: $E$ has a limit point. Use the fact that there is a sequence with distinctive values $\subset E$ converges to this point. We need to guarantee the index is increasing.
    Denote the limit point be $x\in E$ . Then $\exist \{x_{n}\}\sub E, s.t. \,x_n\neq x_m$ for $m\neq n$ and $x_n\to x$ . Since $E=\{p_n\}_{n=1}^{\infty}$ , choose the smallest $n_1$ s.t. $p_{n_1}=x_1$ .
    If $x_2\in \{p_n\}_{n=n_1+1}^{\infty}$ , choose the smallest $n_2$ s.t. $p_{n_2}=x_2$ .
    If $x_2\notin \{p_{n}\}_{n=n_1+1}^{\infty}$ , we drop $x_2$ ,…, must exist $x_k\in \{p_n\}_{n=n_1+1}^{\infty}$ (Because $\{x_n\}$ contains infinitely many elements and $\{p_n\}_{n=n_1+1}^{\infty}$ only deletes finitely many elements), we can then choose the smallest $n_2$ , s.t. $p_{n_2}=x_k$ .
    We can thus get $\{p_{n_k}\}\sub \{x_n\}$ and $p_{n_k}\to x\in E$ .
    Subcase2: $E$ has no limit point. We show that this cannot happen (Use the property of compactness)
    Possibility 1: $q\in E$ . Since $q$ is not a limit point of $E$ , then $\exist$ nbhd $N_{r_q}(q)$ such that $N_r(q)\cap E=\{q\}$ .
    Possibility 2: $q\notin E$ . Since $q$ is not a limit point of $E$ , then $\exist N_{r_q}(q)$ does not contain any point in $E$ , i.e. $N_{r_q}\cap E=\empty$ .
    Obviously, $\cup_{q\in K} N_{r_{q}}(q)$ is an open covering of $K$ , given $K$ is compact, then there exists a finite set $\{q_1,...,q_n\}$ such that $E\subset K \subset\cup_{i=1}^n N_{r_i}(q_i)$ . Hence $\cup_{i=1}^n N_{r_i}(q)\cap E=E$ . But $\cup_{i=1}^n N_{r_i}(q)\subset\{q_1,...,q_n\}$ is finite, so $E$ is a finite set, a contradiction.
- Hw6 Q2: Sequentially Compactness $\Rightarrow$ Compactness
  - A partial proof, the open covering must has the form of $\cup_{i=1}^{\infty} G_{n}$ (the set is countable)
  Argue by contradiction. Suppose no finite subcovering of $\cup_{i=1}^{\infty} G_n$ $\cup_{i=1}^n G_n$ can cover $K$ . Then let $F_n=\cup_{k=1}^n G_k,\forall n\geq 1$ . Then $F_n$ is increasing as $n$ increases, $K \not \subset F_n$ , so $\exist p_n\in K, p_n\notin F_n$ (i.e. $p_n\in F_n^c$ ). Consider this sequence of $\{p_n\}\subset K$ , given $K$ is sequentially compact, $\exist$ subsequence $\{p_{n_k}\}_{k=1}^{\infty}$ such that $p_{n_k}\to$ some $p_{\infty}\in K$ .
  Since $F_n$ is increasing, then $F_n^c$ is decreasing. So $\forall m\geq n, p_m\in F_m^c\subset F_n^c$ . So $\forall$ fixed $n\geq 1$ , let $k'=min\{n_k\geq n\}$ , then $p_{n_k}\in F_n^c$ for all larger $k$ s. Since $F_n$ as the union of open sets is open, $F_n^c$ is closed. Because there is a sequence $\{p_{n_k}\}_{k=k'}^{\infty}\subset F_n^c$ converges to $p_{\infty}$ , then $p_{\infty}$ is either a limit point of $F_n^c$ or a point in $F_n^c$ (or both). Given $p_{\infty}$ is closed, we must have $p_{\infty}\in F_n^c, \forall n\geq 1$ .
  $p_{\infty}\in \cap_{n=1}^{\infty} F_n^c=(\cup_{n=1}^{\infty} F_n)^c=(\cup_{i=1}^{\infty}G_n)^c\subset K^c$
  Contradict with $p_{\infty}\in K$ .

Closed subset of compact set is compact (Hw 6 Q7 & Quiz )
- Use sequential compact
- Useful fact: $A$ is closed $\Leftrightarrow$ $\forall$ convergent sequence $\{x_n\}_{n=1}^{\infty}\subset F$ , we have $\lim_{n\to \infty} x_n\in A$ . (Hw 5 Q1)

Let $\{K_{\alpha}\}_{\alpha \in A}$ be a family of compact set of $X$ . Suppose $\forall$ finite subset $A'$ of $A$ , $\cap_{\alpha \in A'} K_{\alpha}\neq \empty$ , then $\cap_{\alpha \in A} K_{\alpha}\neq \empty$ .
Argue by contradiction. Suppose $\cap_{\alpha \in A} K_{\alpha}=\empty$ . Do complement: $\cup_{\alpha \in A} K_{\alpha}^c=(\empty)^c=X$ . Fix $\alpha_1\in A$ , then $K_{\alpha_1}\subset \cup_{\alpha\in A} K_{\alpha}^c$ . Observe that $K_{\alpha}^c$ is open (because $K_{\alpha}$ is compact, $K_{\alpha}$ is closed), so $\cup_{\alpha\in A} K_{\alpha}^c$ is an open covering of $K_{\alpha_1}$ . Since $K_{\alpha_1}$ is compact, then $\exist \alpha_2,...,\alpha_n\in A$ , s.t. $K_{\alpha_1}\subset\cup_{i=2}^n K_{\alpha_i}^c$ . Do complement again: $\cap_{i=2}^n K_{\alpha_i}\subset K_{\alpha_1}^c$ . Thus $\cap_{i=2}^n K_{\alpha_i}\cap K_{\alpha_1}=\empty,$ contradicting the assumption.

Let $\{K_n\}_{n=1}^{\infty}$ be a sequence of non-empty compact set such that $K_{n+1}\subset K_n$ (decreasing), Then $\cap_{n=1}^{\infty} K_n\neq \empty$ .
- Application: Nested Interval Theorem (Hw 6 Q6)

6. Connected Sets

Definition: Let $A,B\subset$ metric space $X$ , we say $A$ and $B$ are separated if $\bar{A}\cap B=\empty$ and $\bar{B}\cap A=\empty$ . We say $E\subset X$ is connected if $E$ is not a union of two nonempty separated sets, i.e., $\not\exist$ nonempty $A,B\subset E$ , s.t., (1) $E=A\cup B$ ; (2) $A$ and $B$ are separated.

Basic facts
- Basic facts 1: Equivalent definitions
  - $E$ is connected
  - $\not \exist$ open sets $O_1\&O_2\subset X$ , s.t. (1) $E=(E\cap O_1)\cup(E\cap O_2)$ ; (2) $E\cap O_1 \& E\cap O_2$ are nonempty; (3) $E\cap O_1\cap O_2=\empty$ .
  - $\not\exist$ closed sets $F_1\& F_2\subset X$ , s.t. (1) $E=(E\cap F_1)\cup (E\cap F_2)$ ; (2) $E\cap F_1\& E\cap F_2$ are nonempty; (3) $E\cap F_1\cap F_2=\empty$ .
  - Proof (a) $\Rightarrow$ (c)
    Argue by contradiction. Suppose $\exist$ closed $F_1 \&F_2 \subset X$ , s.t. $E=(E\cap F_1)\cup (E\cap F_2)$ where $E\cap F_1$ and $E\cap F_2$ are nonempty and $E\cap F_1\cap F_2=\empty$ .
    We can let $A=E\cap F_1, B=E\cap F_2$ , then $A$ and $B$ are both non-empty and $E=A\cup B$ . We only need to show $\bar{A}\cap B=\empty$ and $\bar{B}\cap A=\empty$ .
    Since $A\subset F_1$ and $F_1$ is closed, by ole result, $\bar{A}\subset F_1$ (because $\bar{A}$ is the smallest closet set containing $A$ ). Similarly, $\bar{B}\subset F_2$ .
    So $\bar{A}\cap B \subset F_1\cap B=E\cap F_1\cap F_2=\empty$
    $\bar{B}\cap A\sub F_2\cap A=F_2\cap E\cap F_1=\empty$
    Hence $A$ and $B$ are separated, contradicting with $E$ is connected.
  - Proof (c) $\Rightarrow$ (a)
    Argue by contradiction. Suppose $\exist$ nonempty $A,B\subset E$ ,s.t. $E=A\cup B$ and $\bar{A}\cap B=\empty=\bar{B}\cap A$ . Take $F_1=\bar{A},F_2=\bar{B}$ .
    Then $F_1\cap E=\bar{A}\cap (A\cup B)=(\bar{A}\cap A) \cup(\bar{A}\cup B)=A \neq \empty$
    $F_2\cap E=\bar{B}\cap (A\cup B)=B\neq \empty$
    and $(E\cap F_1)\cup (E\cup F_2)=A\cup B=E$ .
    $E\cup F_1\cup F_2=A\cup \bar{B}=\empty$ , contradicting to the assumption.
  - (a) $\Leftrightarrow$ (b) see Hw 7 Q2.
- Basic facts 2: Empty set $\empty$ is connected.
- Basic facts 3: If $X$ is connected, then any $E\subset X$ which is both open and closed must be either $X$ or $\empty$ . (Quiz 6)
  Suppose $\exist$ $E$ open & closed and $E\neq X, E\neq \empty$ . Then $E^c$ is also open & closed and $E^c\neq X, E^c\neq \empty$ . Notice that $X=E\cup E^c, \bar{E}\cup E^c=E\cup E^c=\empty, E\cup \bar{E^c}=E\cup E^c=\empty$ , so $X$ is not connected.
- Basic facts 4: Any interval $I\subset R$ is connected.
- Basic facts 5: A convex set is connected. (Hw7 Q3)

Hw 7 Q1: Suppose $E \subset$ metric space $X$ is connected. (i) Prove that $\bar{E}$ is also connected. (ii)Find a counter-example showing that $\dot{E}$ is not connected.
- Key construction: $C=A \cap E, D=B\cap E$ , argue $E$ is not entirely contained in $C$ .

Hw 7 Q2: Prove the equivalent definition of connected set using open set: $E \subset$ metric space $X$ is connected $\Leftrightarrow$ $\nexists$ open sets $O_1$ and $O_2$ in $X$ such that $E\cap O_1\cap O_2=\empty$ . $E\cap O_1$ and $E\cap O_2$ are nonempty, and $E=(E\cap O_1)\cup (E\cap O_2)$ .
- Key construction: “ $\Rightarrow$ ”: $A=E\cap O_1, B=E\cap O_2$ ; “ $\Leftarrow$ ” $O_1=\bar{A}^{c},O_2=\bar{B}^{c}$ .
- “ $\Rightarrow$ ”
  Suppose $\exist O_1,O_2$ such that $(E\cap O_1)\cup (E\cap O_2)=E; E\cap O_1$ and $E\cap O_2$ both non-empty; $E\cap O_1\cap O_2=\empty$ . Let $A=E\cap O_1,B=E\cap O_2$ , then $A,B$ both non-empty and $E=A\cup B$ . We want to show $\bar{A}\cap B=\empty=A\cap \bar{B}$ .
  For any $p\in \bar{A}$ :
  (1) $p\in A=E\cap O_1$ . Since $E\cap O_1\cap O_2=\empty$ , this $p\notin B$ .
  (2) $p\notin E\cap O_1$ but $p\in (E\cap O_1)'$ . Suppose this $p\in B=E\cap O_2$ , then we should have $p\in O_2$ . But given $p$ is a limit point of $E\cap O_2,$ $\forall N_r(p),$ $\exist q\in N_r(p),q\in E\cap O_1$ $\exist q\in N_r(p),q\in E\cap O_2$ . Since $E\cap O_1\cap O_2=\empty$ , this $q\notin O_2$ . Hence $p$ is not an interior point of $O_2$ , contradict with $O_2$ is open. Thus we must have $p\notin B$ .
  Similarly, we have $\bar{B}\cap A=\empty$ , contradict with $E$ is connected.
- “ $\Leftarrow$ ”
  Suppose $\exist$ non-empty $A,B$ such that $E=A\cap B$ and $\bar{A}\cap B=\empty=\bar{B}\cap A$ .
  Let $O_1=\bar{A}^c, O_2=\bar{B}^c$ .
  Since $\bar{A}\cap B=\empty$ and $\bar{B}\cap A=\empty$ , we have $B\subset \bar{A}^c, A\subset \bar{B}^c$ .
  Then $E\cap O_1=(A\cup B)\cap \bar{A}^c=(A\cap \bar{A}^c)\cup (B\cap \bar{A}^c)=B$
  $E\cap O_2=(A\cup B)\cap \bar{B}^c=(A\cap\bar{B}^c)\cup(B\cap\bar{B}^c)=A$
  are both non-empty, and $(E\cap O_1)\cup(E\cap O_2)=A\cup B= E$ ; $E\cap O_1\cap O_2=B\cap\bar{B}^c=\empty$ , contradiction.

Hw 7 Q3: Prove that a convex set is connected.
- Key construction: Take $a\in A, b\in B$ , let $A_0=\{t\in[0,1]|(1-t)a+tb\in A\}$ , $B_0=\{t\in[0,1]|(1-t)a+tb]\in B\}$ , use $A_0$ and $B_0$ to reach the contradiction that $[0,1]$ is not connected.
- Useful fact: $x\in \bar{A}\Leftrightarrow \exist \{x_n\},s.t.\,x_n\to x$

Quiz 6: Let $X$ be a connected metric space. Suppose $E\subset X$ is both open and closed in $X$ . Prove that $E$ is either $X$ or $\empty$ .
- Key observation: $X=E\cup E^{c}$ . And use the fact that $E$ open then $E^c$ closed,vice versa

7. Limit of Functions

Definition: Let $p\in (Dom(f))'$ . We say the limit of $f(x)$ as $x\to p$ is a point $q\in Y$ , if $\forall \varepsilon>0$ , $\exist \delta>0$ , then $d_{Y}(f(x),q)<\varepsilon$ whenever $0<d_{X}(x,p)<\delta, x\in Dom(f)$ .
- $p$ may not in $Dom(f)$ , and $\lim_{x\to p} f(x)$ in general has nothing to do with $f(p)$ .
- If $p\notin (Dom(f))'$ , don’t talk about $\lim_{x\to p} f(x)$ , because no body in the domain is approaching it.
- $0<d_{X}(x,p)<\delta$ , 去心邻域

Cauchy’s criterion
- We say that a real-valued function $f$ defined near $c\in R$ satisfies Cauchy’s conition as $x\to c$ if $\forall \varepsilon>0,\exist\delta>0$ , such that $|f(x)-f(y)|<\varepsilon$ , whenever $0<|x-c|<\delta,0<|y-c|<\delta$ .
- We say that a real-valued function $f$ defined on $(a,\infty)$ satisfies Cauchy’s condition as $x\to \infty$ if $\varepsilon>0,\exist M$ such that $|f(x)-f(y)|<\varepsilon$ as long as $x,y\in (a,\infty)$ , $x>M$ , $y>M$ .

Basic facts
- $\lim_{x\to p} f(x)=q \Leftrightarrow \forall \{p_{n}\}_{n=1}^{\infty} \subset Dom(f)\,\,s.t.\,\, p_n\to p, p_n\neq p, \lim_{n\to \infty} f(p_n)=q$ . (化整为零)
  - $\Rightarrow$
    Since $\lim_{x\to p} f(x)=q$ , then $\forall \varepsilon>0, \exist \delta>0, s.t. \,\,d_Y(f(x),q)<\varepsilon$ , as long as $0<d_X(p_n,p)<\delta$ . Since $\lim_{n\to \infty} p_n=p$ , $p_n\neq p$ , then $\exist N_{\delta}\,\,s.t.\,\, d_X(p_n,p)<\delta,$ whenever
    Hence $\forall \varepsilon>0, \exist N_{\delta}, s.t.$ as long as $n\geq N_{\delta}$ , we have $d_X(p_n,p)<\delta, d_Y(f(p_n),q)<\varepsilon$ , thus $\lim_{n\to \infty} f(p_n)=p$ .
  - $\Leftarrow$
    Suppose not. Then $\exist$ bad $\varepsilon_0>0, s.t. \forall \delta>0,\exist$ bad $x_{\delta}\in Dom(f)$ , $0<d_X(x_{\delta},p)<\delta$ but $d_Y(f(x_\delta),q)>\varepsilon_0$ .
    Take $\delta=1,\frac{1}{2},\frac{1}{3},...$ and let $p_n=x_{\delta}$ , then $0<d_X(p_n,p)<\frac{1}{n}$ , hence $\{p_n\}_{n=1}^{\infty}\subset Dom(f)$ $,\lim_{n\to p} p_n=p$ $\{p_n\}_{n=1}^{\infty}\subset Dom(f)$ $\lim_{n\to p} p_n=p$ and $p\neq p$ . But $d_Y(f(p_n),q)>\epsilon_0,$ which implies $\lim_{n\to \infty} f(p_n)\neq q$ .
  - Remark: $\lim_{x\to p} f(x)\neq q \Leftrightarrow$ $\exist$ bad $\{p_n\}_{n=1}^{\infty} \subset Dom(f)$ , $p_n \to p,p_n\neq p$ , s.t. $\lim_{n\to \infty} f(p_n)\neq q$ $\Leftrightarrow$ $\exist$ bad $\varepsilon_0$ and bad sequence $\{p_n\}_{n=1}^{\infty} \subset Dom(f)$ , $p_n\to p, p_n\neq p,$ and $d_Y(f(p_n),q)\geq \varepsilon_0, \forall n$
- Addition, Multiplication and Quotient Rules for limits of real & complex-valued function

8. Continuity of Functions

Definition: Let $f:Dom(f)\subset X\to Y$ . We say $f$ is continuous at $p\in X$ if: (1) $p\in Dom(f)$ ; (2) $\forall \varepsilon>0,\exist \delta>0$ , s.t. $d_{Y}(f(x),f(p))<\varepsilon$ whenever $d_{X}(x,p)<\delta$ , $x\in Dom(f)$ .
- $p$ may or may not $\in (Dom(f))'$ . If $p\notin (Dom(f))'$ , then $f$ is automatically continuous at $p$ , because $p$ is isolated and nobody in the domain is approching it except itself. If $p\in (Dom(f))'$ , then $f$ is continuous $\Leftrightarrow \lim_{x\to p} f(x)=f(p)$ .
- If $f$ is continuous at every $p\in Dom(f)$ , then we say $f$ is a continuous function.

Main Convenience: $\lim_{x\to x_{0}} f(x)=f(x_0)$ & $\lim_{n\to \infty} f(x_n)=f(x_{\infty})$ . That is, the transformation $f$ perserves the convergence.

Basic Facts
- (Composite function): If $f$ is continuous at $p\in Dom(f)$ , and $g$ is continuous at $f(p)$ , then $g\circ f$ is continuous at $p$ . In particular, if $f$ and $g$ are continuous functions, then so is $g\circ f$ .
  Since $g$ is continuous at $f(p)$ , we have $\forall \varepsilon>0,\exist \delta>0,s.t.$
  $d_{Z}(g(y),g(f(p)))<\varepsilon$ if $d_{Y}(y,f(p))<\delta$
  Since $f$ is continuous at $p$ , for this $\delta$ , there exists $r>0$ , s.t.
  $d_{Y}(f(x),f(p))<\delta$ if $d_{X}(x,p)<r$
  Combine them together, we have $\forall \varepsilon>0$ , $\exist r>0$ , s.t whenever $d_{X}(x,p)<r$ ,
  we have $d_{Y}(f(x),f(p))<\delta$ , and consequently, $d_{Z}(g(f(x)),g(f(p)))<\delta$ . Hence $g\circ f$ is continuous at $p$ .
- (Pre-image)A mapping $f$ of a metric space $X$ into a metric space $Y$ is continuous on $X$ . if and only if $f^{-1}(V)\equiv\{x\in X|f(x)\in V\}$ is open in $X$ for every open set $V$ in $Y$ .
  - $\Rightarrow$
    Suppose $f$ is continuous on $X$ and $V$ is an open set in $Y$ . We have to show that every point of $f^{-1}(V)$ is an interior point of $f^{-1}(V)$ . So, suppose $p\in X$ and $f(p)\in V$ . Since $V$ is open, there exists $\varepsilon>0$ such that $N_{\varepsilon}(f(p))\subset V$ . Since $f$ is continuous at $p$ , there exists $\delta>0$ such that $f(q)\in N_{\varepsilon}(f(p))\subset V$ if $q \in N_{\delta}(p)$ . Thus $N_{\delta}(p)\in f^{-1}(V)$ .
  - $\Leftarrow$
    For all $p\in X$ , $\forall \varepsilon$ , let $V=N_{\varepsilon}(f(p))$ which is open. Then $f^{-1}(V)$ is open. Since $p\in f^{-1}(V)$ , there exists $N_{\delta}(p)\subset f^{-1}(V)$ . Hence as long as $d_X(p,q)<\delta$ , we have $f(q)\in N_{\varepsilon}(f(p))$
  - Corollary: A mapping $f$ of a metric space $X$ into a metric space $Y$ is continuous if and only if $f^{-1}(C)$ is closed in $X$ for every closed set $C$ in $Y$ .
    Since $(f^{-1}(E))^C=f^{-1}(E^C)$ for every $E\subset Y$
- Continuity and Compactness
  - If $f:X\to Y$ is continuous and $X$ is compact, then $f(X)$ is also compact.
    Motivation: is it true that $f(O)=O$ and $f(C)=C$ ? In general, No.
    Proof 1: Sequential Compact
    We just need to show $f(X)$ is sequentially compact, that is $\forall \{y_n\}_{n=1}^{\infty} \subset f(X)$ , exists a convergent subsequence $\{y_{n_k}\}_{k=1}^{\infty}$ such that $y_{n_k}\to y_{\infty}\in f(X)$ .
    Consider $\forall \{y_n\}_{n=1}^{\infty} \subset f(X)$ , we can back out a corresponding sequence $\{x_n\}_{n=1}^{\infty} \subset X$ . That is, there exists $\{x_n\}_{n=1}^{\infty}$ such that $y_n=f(x_n)$ . Since $\{x_n\}_{n=1}^{\infty}$ and $X$ is compact, there exists a convergent subsequence $\{x_{n_k}\}_{k=1}^{\infty}$ such that $x_{n_k}\to x_{\infty}\in X$ . Since $f$ is continous, $\lim_{k\to \infty} y_{n_k}=\lim_{k\to \infty} f(x_{n_k})=f(x_{\infty})\in f(X)$
    Proof 2: Finite Open Covering
    Let $\{V_{\alpha}\}$ be an open cover of $f(X)$ . Since $f$ is a continuous function, we have each preimage $f^{-1}(V_{\alpha})$ is open. Consequently, $\{f^{-1}(V_{\alpha})\}$ is an open cover of $X$ . Given $X$ is compact, there are finite many indices, say $\alpha_1,...,\alpha_n$ such that
    $X\subset f^{-1}(V_{\alpha_1})\cup ...\cup f^{-1}(V_{\alpha_n})$
    Since $f(f^{-1}(E))\subset E$ for every $E\subset Y$ , this implies that
    $f(X)\subset V_{\alpha_1}\cup...\cup V_{\alpha_2}$
  - (Extreme Value Theorem) If $f:X\to R$ is continuous and $X$ is compact, then $\exist p,q\in X,s.t.\,\,f(p)=\sup_{x\in X} f(x)$ , $f(q)=\inf_{x\in X} f(x)$ .
    Since $f$ is continuous and $X$ is compact, then $f(X)$ is compact. Since $f(X)\subset R$ , we have $f(X)$ is closed and bounded. Hence $f(X)=\overline{f(X)}$ . In ole Hw, if $E\subset R$ and $E$ is bounded, then $\sup E\in \bar{E}$ . Hence $\sup_{x\in X} f(x)\in f(X)$ .
  - If $f:X\to Y$ is continuous & 1-1 & onto & $X$ is compact, then $f^{-1}:Y\to X$ is also continuous.
    $\forall y_0 \in Y$ , w.t.s $f^{-1}$ is continuous at $y_0$ .
    If $y_0\notin Y'$ , then $y_0$ is isolated, $f^{-1}$ is automatically continuous at $y_0$ (we can always find a proper $\delta$ such that the neighbor hood of $y_0$ only contain $y_0$ itself, so that $d_X(f^{-1}(y),f^{-1}(y))=0$ for all $y\in N_{\delta}(y_0)$ . )
    If $y_0\in Y'$ , then we need to show $\lim_{y\to y_0}f^{-1}(y)=f^{-1}(y_0)$ . Suppose not, then $\exist$ bad $\delta_0$ and bad sequence $\{y_n\}_{n=1}^{\infty}$ such that $y_n\to y_0$ but $d_X(f^{-1}(y_n),f^{-1}(y_0))\geq \delta_0$ . Since $f$ is 1-1 and onto, then $\exist$ $\{x_n\}_{n=1}^{\infty}$ such that $f(x_n)=y_n$ . (or equivalently, $x_n=f^{-1}(y_n)$ ). Since $X$ is compact, $\exist$ subsequence $\{x_{n_k}\}_{k=1}^{\infty}$ such that $x_{n_k}\to x_{\infty} \in X$ . Since $f$ is continuous, $\lim_{k\to \infty} f(x_{n_k})=f(x_\infty)$ . But $\lim_{k\to \infty} f(x_{n_k})=\lim_{k\to \infty} y_{nk}=y_0$ . Hence $x_{\infty}=f^{-1}(y_0)$ . But $d_X(x_{n_k},x_{\infty})=d_X(f^{-1}(y_{n_k}),f^{-1}(y_0))\geq \varepsilon_0$ , contradict with $x_{n_k}\to x_{\infty}$ .
    Remark: If $X$ not compact, then this fact may fail.
  - (Uniformly Continuous): If $f:X\to Y$ is continuous & $X$ is compact, then $f$ is uniformly continuous on $X$ .
    Definition: Let $f:X\to Y$ , we say $f$ is uniformly continuous on $X$ if $\forall \varepsilon>0,\exist \delta>0,s.t.\,\,d_Y(f(x),f(y))<\varepsilon,$ whenever $x,y\in X,d_X(x,y)<\delta$ .
    Proof 1: Finite Open Covering
    Fix $\varepsilon$ . Since $f$ is continuous, for each point $p\in X$ , we can find a $\delta(p)$ , such that
    $d_{Y}(f(q),f(p))<\frac{\varepsilon}{2}$ if $d_{X}(q,p)<\delta(p)$
    Then $\cup_{p\in X} N_{\frac{1}{2}\delta(p)}(p)$ is an open cover of $X$ . Since $X$ is finite, there is a finite set of points $p_1,...,p_n$ in $X$ such that
    $X\subset N_{\frac{1}{2}\delta(p_1)}(p_1)\cup...\cup N_{\frac{1}{2}\delta(p_n)}(p_n)$
    We choose $\delta=\frac{1}{2}\min\{\delta(p_1),...,\delta(p_n)\}$ (The min exists due to finiteness, this is one point where finiteness of the covering, inherent in the definition of compactness, is essential). Now let $p,q$ be points in $X$ such that $d(p,q)<\delta$ .
    Since $X\subset N_{\frac{1}{2}\delta(p_1)}(p_1)\cup...\cup N_{\frac{1}{2}\delta(p_n)}(p_n)$ , $\exist$ $i$ such that $p\in N_{\delta(p_i)}(p_i)$ .
    Then $d(q,p_i)\leq d(q,p)+d(p,p_i)\leq \delta+\frac{1}{2} \delta(p_i)\leq\delta(p_i)$
    Hence $d(f(p),f(q))\leq d(f(p),f(p_i))+d(f(p_i),f(q))\leq\frac{\varepsilon}{2}+\frac{\varepsilon}{2}=\varepsilon$
    Proof 2: Sequential Compact
    Suppose not, then $\exist$ bad $\varepsilon_0>0,$ such that $\forall \delta>0,$ $\exist$ bad pair $(x_{\delta},y_{\delta})\in X, d_{X}(x_{\delta},y_{\delta})<\delta$ but $d_{Y}(f(x_{\delta}),f(y_{\delta})\geq \varepsilon_0$ .
    Take $\delta_{n}=\frac{1}{n}$ , and let $x_n=x_{\delta_n},y_n=y_{\delta_n}$ , then we have
    $d_{X}(x_n,y_n)<\frac{1}{n}$ but $d_{Y}(f(x_n),f(y_n))\geq \varepsilon_0$
    Since $X$ is compact, then $\exist$ subseq $\{x_{n_k}\}$ and $\{y_{n_k}\}$ (do they have the same index?) such that $x_{n_k}\to x_{\infty}\in X$ and $y_{n_k}\to y_{\infty}\in X$ .
    $\{x_{n_k}\}$ and $\{y_{n_k}\}$ fly together,
    $d(x_{\infty},y_{\infty})\leq d(x_{\infty},x_{n_k})+d(x_{n_k},y_{n_k})+d(y_{n_k},y_{\infty})\to 0$ , so $x_{\infty}=y_{\infty}$ .
    Since $f$ is continuous, we should have $f(x_{n_k})\to f(x_{\infty}), f(y_{n_k})\to f(y_{\infty})$ , But $f(x_{\infty})=f(y_{\infty})$ , they approach the same limit,
    $d(f(x_{n_k}),f(y_{n_k}))\leq d(f(x_{n_k}),f(x_{\infty}))+d(f(x_{\infty}),f(y_{\infty}))+d(f(y_{\infty}),f(y_{n_k})) \to 0$ , contradcit with $d(f(x_n),f(y_n))\geq \varepsilon_0$ .
    Hw 8 Q3: Replace $[a,b]$ by $[a,\infty)$ . Show that $f$ is uniformly continuous in the following two scenarios:
    $f'$ is bounded.
    $\lim_{x\to\infty} f(x)$ exists as a finite # (Divide into $[a,M]$ and $(M,\infty)$ )
    Hw 8 Q4: Let $f(x)$ be real-valued, uniformly continuous on finite interval $(a,b)$ . Prove that $\lim_{x\to b^{-}}f(x)$ and $\lim_{x\to a^{+}}f(x)$ exists as finite #. (Proving the existence of limits without knowing the exact value——Cauchy’s criterion; Treating $\{f(x_n)\}_{n=1}^{\infty}$ as a sequence.)
  - Hw 8 Q5: Let $f(x)$ and $g(x)$ be continuous on $[0,1]$ with $f(x)>g(x),\forall x\in[0,1]$ . Prove that $\exist C>1$ , s.t. $f(x)>Cg(x),\forall x\in [0,1]$ .
- Continuity and Connectedness
  - Let $f:X\to Y$ be continuous and $E\subset X$ be connected. Then $f(E)$ is connected
    Proof1: Original definition of nonempty separated set
    Proof2: Equivalent definition with open set
    Suppose not, then $\exist$ open $O_1 \& O_2 \subset Y$ such that
    $f(E)=(f(E)\cap O_1) \cup (f(E)\cap O_2)$
    $f(E)\cap O_1 \cap O_2=\empty$
    $f(E)\cap O_1 \neq\emptyset$ , $f(E)\cap O_2 \neq \emptyset$ .
    Let $V_1=f^{-1}(O_1),V_2=f^{-1}(O_2)$ . Since $f$ is continuous and $O_1\&O_2$ are open, $V_1 \&V_2$ are open. Observe $f(E)\subset O_1 \cup O_2$ , hence
    $E=f^{-1}(f(E))\subset f^{-1}(O_1\cup O_2)=f^{-1}(O_1)\cup f^{-1}(O_2)=V_1\cup V_2$ .
    Hence $E\subset (V_1\cup V_2)\cap E=(V_1 \cap E)\cup(V_2\cap E)\subset E$ .
    This implies that $E=(V_1\cap E)\cup (V_2\cap E)$
    Then we show $V_1\cap E\neq \emptyset$ and $V_2\cap E \neq \emptyset$ . Since $f(E)\cap O_1 \neq \emptyset$ , then $\exist e\in E,s.t. f(e)\in O_1 \Rightarrow e\in f^{-1}(O_1)=V_1$ . Hence $e\in E\cap V_1$ .
    
    $V_2\cap E \neq \emptyset$ can be proved similarly.
    Finally, we need to show $V_1\cap E\cap V_2=\emptyset$ . Suppose not, then $\exist$ $e\in V_1\cap E\cap V_2$ , such that $f(e)\in O_1 \cap O_2\cap f(E)$ contradiction.
  - (Intermediate Value Theorem): Let $f:[a,b]\to R$ be continuous. Let $m=\min_{[a,b]} f$ , $M=\min_{[a,b]} f$ . If $m<M$ , then $\forall c\in (m,M)$ , $\exist d=[a,b]$ , s.t. $f(d)=c$ .
    Corollary: $f([a,b])=[m,M]$ .
- Continuity and Dense
  - Hw 7 Q5: If $f:X\to Y$ is a continuous mapping , prove that $f(\bar{E})\subset \overline{f(E)}$
    Use definition.
  - Hw 7 Q7: Let $f$ and $g$ be continuous mappings of a metric space $X$ into a metric space $Y$ , and let be a dense subset of $X$ . Prove that $f(E)$ is dense in $f(X)$ . If $g(p)=f(q)$ for all $p\in E$ , prove that $g(p)=f(p)$ for all $p\in X$ .
    Useful fact: $\forall x\in B, \exist \{x_n\}\subset A,s.t.\,\,x_n\to x \Leftrightarrow$ $A$ is dense in $B$ . Combine with $\lim_{n\to \infty} f(x_n)=f(x_{\infty})$ get full-fledged.
    Key construction: $h(x)=f(p)-g(p)$ .
  - Hw 8 Q1: Prove that the Dirichelt’s function (equals 1 if $x$ is rational and equals 0 otherwise) is not continuous at any $x_0\in R$ . (Hint: $Q$ is dense, the equivalent definition of limit points)
- Continuity and Convexity (Hw 8 Q10)

9. Discontinuity of Functions

Discontinuity of Monotone Functions
- If $f(x_0+o) \& f(x_0-o)$ exist as finite numbers and not equal, or $f(x_0+o)=f(x_0-o)$ but $\neq$ $f(x_0)$ , then we say that $f$ has discontinuity of 1st kind at $x_0$ .
- Fact: Suppose $f:(a,b)\to R$ is increasing, then $\forall x_0\in(a,b), \lim_{x\to x_0^{-}}f(x)$ and $\lim_{x\to x_0^{+}} f(x)$ exists as finite numbers, More precisely, $\sup_{a<x<x_0} f(x)=\lim_{x\to x_0^{-}} f(x)\leq \lim_{x\to x_0^{+}} f(x)=\inf_{x_0<x<b}f(x)$ . Furthermore, if $a<x<y<b$ , then $\lim_{t\to x{+}} f(t)\leq \lim_{t\to y^{-}}f(t)$
  - Proof:
    Denote $\inf_{x_0<x<b} f(x)=B$ . Then $\forall \varepsilon>0,$ $\exist x'\in(x_0,b)$ , s.t. $f(x')<B+\varepsilon$
    Since $f$ is increasing, then on $x\in(x_0,x')$ , $B\leq f(x)\leq f'(x)<B+\varepsilon$ .
    Hene $|f(x)-B|<\varepsilon$ , $\forall x>x_0$ and $x-x_0<x'-x_0$ ( $\delta$ )
    Hence $\lim_{x\to x^{+}} f(x)$ exists & $=B$ . Similarly, $\lim_{x\to x^{-}} f(x)$ exists & $=A$ .
    To prove $\lim_{x\to x_{0}^{-}} f(x)\leq \lim_{x\to x_0^{+}} f(x)$ , only need to show
    $\sup_{a<x<x_0} f(x)\leq \inf_{x_0<x<b} f(x)$
    Take $x\in (a,x_0)$ and $x'\in(x_0,b)$ , then $f(x)\leq f(x')$
    Fix $x'$ , take sup, $\sup_{a<x<x_0} f(x)\leq f(x')$
    Since $x'$ is arbitrary, we have $\sup_{a<x<x_0} f(x)\leq \inf_{x_0<x<b} f(x)$
  Moral of the story: monotone functions have only simple discontinuities. Their continuities can only come from ``jumps''
- Theorem: If $f$ is increasing on $(a,b)$ , then the set $D$ of discontinuous point of $f$ is at most countable.
  Let $D=\{x\in(a,b)|f(x-)<f(x+)\}$
  Define a mapping $T_1:x\in D\to (f(x-),f(x+))$ . (i.e. output an interval).
  Define $T_2:(f(x-),f(x+))\to r_x\in Q,r_x\in (f(x-),f(x+))$
  Then consider the composite $T_2\circ T_1$ , essentially, for every point $x\in D$ , we associate a rational number $r(x)$ such that $f(x-)<r(x)<f(x+)$ . We want to show this mapping $r(x)$ is 1-1.
  If $x\neq y$ , WLOG, let $x<y$ , then $f(x+)\leq f(y-)$ , hence
  $(f(x-),f(x+)) \cap (f(y-),f(y+))=\empty$
  Hence $r(x)\neq r(y)$ . Hence $D\sim r(D)\in Q$ is at most countable.

10. Differentiation

Differentiation of Multivariable & Vector-valued functions
- Motivation: Generalize from single-variable real-valued function
  The difficulty is that we can not have vector divided by a vector.
  - Step 1: single-variable real-valued function
    If $f$ is a real-valued function with domain $(a,b)\subset R$ and if $x\in (a,b)$ , then $f'(x)$ is usually defined to be the real number
    $f'(x)=\lim_{h\to 0} \frac{f(x+h)-f(x)}{h}$
    provided this limit exists. Thus
    $f(x+h)-f(x)=f'(x)h+r(h)$
    where the “remainder” is small, in the sense that it converges to 0 faster than $h$
    $\lim_{h\to 0}\frac{r(h)}{h}=0$
    We call $r(h)$ the higher order infinitisemal of $h$ , denoted as $o(h)$ . It is any function satisfying $\lim_{h\to 0}\frac{o(h)}{h}=0$ .
    Note that the difference $f(x+h)-f(x)$ can be expressed as the sum of the linear function that takes $h$ to $f'(x)h$ , plus a small remainder. We can therefore regard the derivative of $f$ at $x$ , not as a real number, but as the linear operator on $R$ that takes $h$ to $f'(x)h$ .
  - Step 2: single-variable vector-valued function
    Let us next consider a function $\mathbf{f}$ that maps $(a,b)\subset R$ into $R^m$ . In that case, $\mathbf{f'}(x)$ was defined to be that vector $\mathbf{y}\in R^m$ (if there is one) for which:
    $\lim_{h\to0} \frac{\mathbf{f}(x+h)-\mathbf{f}(x)}{h}=\mathbf{y}$
    We can again rewrite this in the form:
    $\mathbf{f}(x+h)-\mathbf{f}(x)=\mathbf{y}h+\mathbf{r}(h)$
    where $\lim_{h\to 0}\frac{\mathbf{r}(h)}{h}=\mathbf{0}$ . The main term of the RHS is again a linear function of $h$ .
    Thus if $\mathbf{f}: (a,b)\to R^m$ is differentiable at $x$ then $\mathbf{f}'(x)$ is the linear transformation of $R\to R^m$ that satisfies
    $\lim_{h\to 0}\frac{\mathbf{f}(x+h)-\mathbf{f}(x)-\mathbf{f}'(x)h}{h}=0$
- Let $f:O(\text{open in} \,\,R^n)\to R^m,x \in O$ . We say $f$ is differentiable at $x$ if there exists $A_{m\times n}$ such that
$f(x+h)=f(x)+Ah+o(|h|)$
as $h\to 0$ $. \,A$ is called the total derivative of $f$ at $x$ . Notation $f'(x)=A=Df(x)$ .
Alternatively, $\lim_{h\to 0}\frac{|f(x+h)-f(x)-Ah|}{|h|}=0$ .
- Proving differentiability : Play with $o(|h|)$ .
  - Remark: $o(g(h))$ represents a function $p(h)$ satisfying $\lim_{h\to 0} \frac{p(h)}{g(h)}=0$ . The term inside $o(.)$ indicates the decaying speed, not the direction. We always take about the limit when $h\to 0$ , only the function we compared with is different. $o(g(h))$
  - Differntiabilty $\Rightarrow$ Partial differentiability. Relation with Jacobian Matrix & Gradient.
  - Chain rule
    Let $f:O (\text{open in}\,\,R^n)\to R^m$ be differentiable at $c\in O$ . Let $g:Nr(f(c))\in R^m\to R^p$ be differentiable at $f(c)$ . Then $g\circ f$ is differentiable at $c$ . Moreover,
    $D(g\circ f)(c)=D g(f(c))Df(c)$
    Proof
    Since $f$ is differentiable at $c$ , we have
    $f(c+h)=f(c)+Df(c)h+o(|h|),$ as $h\in R^n\to0$ .
    Since $g$ is differentiable at $f(c)$ , we have
    $g(f(c)+l)=g(f(c))+Dg(f(c))l+o(|l|),$ as $l\in R^m\to 0$ .
    Take $l=f(c+h)-f(c)$ , since $f$ is continuous at $c$ , then $l\to 0$ as $h\to 0$
    Hence
    $g(f(c+h))=g(f(c))+Dg(f(c))(Df(c)h+o(|h|))+o(|Df(c)h+o(|h|)|)$
    $=g(f(c))+Dg(f(c))Df(c)h+o(|h|)+o(1)|Df(c)h+o(|h|)|$
    $=g(f(c))+Dg(f(c))Df(c)h+o(|h|)+o(1)O(1)|h|$
  - Continuous partial differentiability $\Rightarrow$ Total differentiability
    It suffices to show componentwise.
    $f_i(c+h)=f(c)+\nabla f_i(c) h+o(|h|)$
    that is $f_i(c+h)-f_i(c)-\nabla f_i(c)h=o(|h|)$
    Use a trick similar to telescoping, we build the bridge to link $f_i(c+h)$ and $f_i(c)$ by creating a series of $\Delta s$ . (minus some term, then plus back). Each time, we eliminate the increment $h$ on one dimension, and use the MVT of a single varible to better characterize the resulting difference.
    $=f_i(c_1+h_1,...,c_n+h_n)-f_i(c_1,c_2+h_2,...,c_n+h_n)$
    $+f_i(c_1,c_2+h_2,...,c_n+h_n)-f_i(c_1,c_2,c_3+h_3,...,c_n+h_n)$
    ……
    $+f_i(c_1,...,c_{n-1},c_n+h_n)-f_i(c_1,...,c_n)-\sum_{k=1}^n \frac{\partial f_i(c_1,...,c_n)}{\partial x_k}h_k$
    Applying MVT on $[c_1,c_1+h_1],[c_2,c_2+h_2],...,[c_n,c_n+h_n]$ ,we have
    $=\frac{\partial f_i(\tilde{c}_1,c_2+h_2,...,c_n+h_n)}{\partial x_1}h_1+\frac{\partial f_i(c_1,\tilde{c_2},c_3+h_3,...c_n+h_n)}{\partial x_2}h_2+...+\frac{\partial f_i(c_1,...,c_{n-1},\tilde{c}_n)}{\partial x_n}h_n$
    $-\frac{\partial f_i(c_1,...,c_n)}{\partial x_1}h_1-\frac{\partial f_i(c_1,...,c_n)}{\partial x_2}h_2-...-\frac{\partial f_i(c_1,...,c_n)}{\partial x_n}h_n$
    $=(\frac{\partial f_i(\tilde{c}_1,...,c_n+h_n)}{\partial x_1}-\frac{\partial f_i(c_1,...,c_n)}{\partial x_1}) h_1+...+(\frac{\partial f_i(c_1,...,c_{n-1},\tilde{c}_n)}{\partial x_n}-\frac{\partial f_i(c_1,...,c_n)}{\partial x_n})h_n$
    Since all the partial derivatives are continuous, the coefficient in front of $h_i$ goes to 0 as $h\to0$ . Hence we have
    $=o(1)h_1+...+o(1)h_n$ $=o(1)\frac{h_1}{|h|}|h|+...+o(1)\frac{h_n}{|h|}|h|$
    Since $-1<\frac{h_i}{|h|}<1$ , they are all $O(1)$ , then we have
    $=o(1)O(1)|h|+...+o(1)O(1)|h|=o(|h|)$
  - HW9 Q6
  - HW10 Q1: Suppose that $f$ is a real-valued function defined on open set $E\subset R^n$ , and that the partial derivatives $D_1f,...,D_nf$ are bounded in $E$ . Prove that $f$ is continuous in $E$ .
  - Hw10 Q2: Let $c=(c_1,c_2)\in$ open set $O\subset R^2$ . Suppose $f:O\to R$ satisifies $\frac{\partial f}{\partial y}$ exists and is continuous and $\frac{\partial f}{\partial x}(c_1,c_2)$ exists. Prove that $f(x,y)$ is differentiable at $c$ .

Mean Value Theorem (Killer’s Instinct)
- Rolle’ s Theorem (Baby MVT): Suppose $f$ is real-valued continuous on $[a,b]$ , differentiable on $(a,b)$ , $f(a)=f(b)$ . Then $\exist c\in(a,b)$ such that $f'(c)=0$ .
- MVT(Lagrange): Suppose $f$ is real-valued continuous on $[a,b]$ , differentiable on $(a,b)$ , then $\exist c\in (a,b),$ such that $\frac{f(b)-f(a)}{b-a}=f'(c)$ . (某处的瞬时速度等于整段的平均速度)
  - Corollary: Suppose $f$ real-valued and differentiable on $(a,b)$ , then (i) If $f'(x)\geq 0$ on $(a,b)$ , then $f$ is increasing on $(a,b)$ ; (ii) If $f'(x)\leq0$ on $(a,b)$ , then $f$ is decreasing on $(a,b)$ . (iii) If $f'\equiv 0$ on $(a,b)$ , then $f\equiv c$ on $(a,b)$ .
- MVT(Cauchy): Let $f(x)$ & $g(x)$ be real-valued continuous on $[a,b]$ , differentiable on $(a,b)$ , then $\exist c\in(a,b)$ , such that $(f(b)-f(a))g'(c)=(g(b)-g(a))f'(c)$ .
- MVT for multivariable vector-valued functions: Suppose $f: O(\text{open}\,\,\subset R^n)\to R^m$ is differentiable on $O$ & assume $O$ is convex, and $||Df(c)_{m\times n}||$ is bounded on $O$ , denoted a bound by $M$ , then $|f(b)-f(a)|\leq M|b-a|$ .
  - Proof
    Trick: Transform a multivariable function into a single variable function
    Fix arbitrary $z\in R^m$ ,
    $g(t)=z\cdot f(ta+(1-t)b)$ ( $\cdot$ represents the dot product), this is a single variable real-valued function. Apply MVT to $g(t)$ on $[0,1]$
    $g(1)-g(0)=g'(c)(1-0)$ , for some $c\in(0,1)$
    $z\cdot f(a)-z\cdot f(b)= z\cdot f'(ca+(1-c)b)(a-b)$
    $|z\cdot (f(a)-f(b))|\leq |z| |f'(ca+(1-c)b)(a-b)|$
    $\leq |z|||f'(ca+(1-c)b)|||a-b|$
    $\leq |z|M|a-b|$
    (Inequalities used: $|Ax|\leq ||A|||x|,|a\cdot b|\leq|a||b|$ )
    Take $z=f(a)-f(b)$ , then
    $|f(a)-f(b)|^2\leq |f(a)-f(b)|M|a-b|$
    $\Rightarrow |f(a)-f(b)|\leq M|a-b|$
  - Corollary:If $Df\equiv0$ on $O$ and $O$ is convex, then we can take $M=0$ and get $f\equiv c$ on $O$ .
- Mixed derivatives
  Let $f:\text{open}\,\,O\subset R^n \to R, f\in C^1(O),$ suppose $\frac{\partial }{\partial x_j}(\frac{\partial f}{\partial x_i})$ exists & is continuous on $O$ . Then the mixed derivative $\frac{\partial}{\partial x_i}(\frac{\partial f}{\partial x_j})$ also exists & $=\frac{\partial}{\partial x_j}(\frac{\partial f}{\partial x_i})$
  - Lemma: Let $f(x,y)$ be real-valued & defined near $(x_0,y_0)\in R^2$ and $\lim_{(x,y)\to(x_0,y_0)}f(x,y)=A$ (finite number). Assume $\forall y\approx y_0$ , $\lim_{x\to x_0} f(x,y)$ exists as a finite number, denoted as $g(y)$ . Then $\lim_{y\to y_0} g(y)=A$ .
    Idea: 一维一维地逼近
    $\lim_{y\to y_0}(\lim_{x\to x_0} f(x,y))=\lim_{(x,y)\to (x_0,y_0)}f(x,y)$ .
    Since $\lim_{(x,y)\to (x_0,y_0)} f(x,y)=A$ , then $\forall \varepsilon>0, \exist \delta>0, s.t.$
    $|f(x,y)-A|<\varepsilon$ whenever $|(x,y)-(x_0,y_0)|<\delta$
    Now fix $y\approx y_0,y\neq y_0,s.t. |y-y_0|<\frac{\delta}{612}$
    Then $|f(x,y)-A|<\varepsilon$ for all $x$ close to $x_0$ . To obaint $g(y)$ , sending $x\to x_0$
    $\lim_{x\to x_0} |f(x,y)-A|<\varepsilon \Rightarrow |g(y)-A|\leq \varepsilon$
    Since $\forall \varepsilon>0,$ exists $\frac{\delta}{612}$ , such that $|g(y)-A|\leq\varepsilon$ as long as $|y-y_0|<\frac{\delta}{612}$ , we have $\lim_{y\to y_0}g(y)=A$ .
  WLOG, assume $f$ is a function of $(x,y)\in O \subset R^2$ .
  What we know: $\frac{\partial}{\partial y}(\frac{\partial f}{\partial x})(c)=\lim_{k\to 0} \frac{\frac{\partial f(c_1,c_2+k)}{\partial x}-\frac{\partial f(c_1,c_2)}{\partial x}}{k}=A$
  Want to show: . $\frac{\partial}{\partial x}(\frac{\partial f}{\partial y})(c)=A$ .
  $\frac{\partial}{\partial x}(\frac{\partial f}{\partial y})(c)=\lim_{h\to 0}\frac{\frac{\partial f(c_1+h,c_2)}{\partial y}-\frac{\partial f(c_1,c_2)}{\partial y}}{h}$
  $=\lim_{h\to 0} \frac{1}{h} [\lim_{k\to 0}\frac{f(c_1+h,c_2+k)-f(c_1+h,c_2)}{k}-\frac{f(c_1,c_2+k)-f(c_1,c_2)}{k}]$
  $=\lim_{h\to 0}\lim_{k\to 0} \frac{1}{hk}[f(c_1+h,c_2+k)-f(c_1+h,c_2)-f(c_1,c_2+k)+f(c_1,c_2)]$
  Use MVT to create a similar form as what we know, eliminate $h$
  - Method 1
    $=\lim_{h\to 0} \lim_{k\to 0} \frac{1}{hk}[(f(c_1+h,c_2+k)-f(c_1,c_2+k))-(f(c_1+h,c_2)-f(c_1,c_2))]$
    $=\lim_{h\to 0}\lim_{k\to 0}\frac{1}{hk} [\frac{\partial f(\tilde{c}_1,c_2+k)}{\partial x}h-\frac{\partial f(\bar{c}_1,c_2)}{\partial x}h]$ . But $\tilde{c}_1$ may not equal $\bar{c_1}$ .
  - Method 2
    Alternatively, define $F(h)=f(c_1+h,c_2+k)-f(c_1+h,c_2)$
    Then $F(h)-F(0)=F'(\tilde{h})h=(\frac{\partial f(c_1+\tilde{h},c_2+k)}{\partial x}-\frac{\partial f(c_1+\tilde{h},c_2)}{\partial x})h$
    Now the equation becomes:
    $\lim_{h\to 0}\lim_{k\to 0} \frac{\frac{\partial f(c_1+\tilde{h},c_2+k)}{\partial x}-\frac{\partial f(c_1+\tilde{h},c_2)}{\partial x}}{k}=\lim_{h\to 0} \frac{\partial }{\partial y}(\frac{\partial f}{\partial x})(c_1+\tilde{h},c_2)$
    Since $\frac{\partial}{\partial y}(\frac{\partial f}{\partial x})$ is continuous, we have
    $\lim_{h\to 0} \frac{\partial }{\partial y}(\frac{\partial f}{\partial x})(c_1+\tilde{h},c_2)=\frac{\partial }{\partial y}(\frac{\partial f}{\partial x})(c)=A$ .
- L’ Hospital’s Rule (Hw9 Q2)
  Suppose $f$ and $g$ are real-valued and differentiable on $(a,b)$ with $g'(x)\neq 0,\forall x\in (a,b)$ and $\lim_{x\to a^{+}}\frac{f'(x)}{g'(x)}=$ finite $l$ .
  - If $\lim_{x\to a^{+}} f(x)=0=\lim_{x\to a^{+}} g(x)$ , then $\lim_{x\to a^{+}} \frac{f(x)}{g(x)}=l$ .
  - If $\lim_{x\to a^{+}} g(x)=\infty$ , then $\lim_{x\to a^{+}}\frac{f(x)}{g(x)}=l$ .

Properties of $f'$
- (First Derivative Test)Suppose $c$ is an interior point of $Dom(f)$ and $f(c)$ is local extreme and $f$ is partially differentiable at $c$ , then $\bigtriangledown f(c)=0$ .
- (Hw9 Q3): The derivative function of a real-valued function also satisfies the Intermediate Value Property: Suppose $f$ is differentiable on $[a,b]$ with $f'(a)\neq f'(b)$ . Then $\forall \lambda$ between $f'(a)$ and $f'(b)$ , $\exist c\in (a,b)$ such that $f'(c)=\lambda$ .
- (Hw9 Q4): $f'$ cannot have simple discontinuities in $(a,b)$ .
- (Hw9 Q5): Let $f: (a,b)\to R$ be differentiable and convex. Prove that $f'$ is continuous on $(a,b)$ . (First show $f'$ is increasing).
- (Hw9 Q7): Let $f:$ open and connected $O\subset R^n\to R^m$ . Suppose $f'(x)=0,\forall x\in O$ .Prove that $f$ is constant in $O$ . (By Multivariable MVT, we have this result holds for convex sets)
- (Hw9 Q11): If $f$ is differentiable on $(a,b)$ and if $f'$ is increasing on $(a,b)$ , then $f$ is convex on $(a,b)$ .
- (Hw10 Q3): Suppose $a\in R$ , $f$ is a twice-differentiable real function on $(a,\infty)$ , and $M_0,M_1,M_2$ are the least upper bounds of $|f(x)|,|f'(x)|,|f''(x)|$ , respectively, on $(a,\infty)$ . Prove that $M_1\leq 4 M_0 M_2$ .
- (Hw10 Q4): Suppose $f$ is twice-differentiable on $(0,\infty),$ $f''$ is bounded on $(0,\infty)$ , and $f(x)\to0$ as $x\to\infty$ . Prove that $f'(x)\to 0$ as $x\to\infty$ .

Taylor’s Theorem
- Motivation: Define $P_n(x)=f(x_0)+f'(x_0)(x-x_0)+...+\frac{f^{(n)}(x_0)}{n!}(x-x_0)^n$ to be the $n^{th}$ order Taylor expansion at $x_0$ , which is a $n^{th}$ order polynomial that shares the first $n$ order derivatives with $f(x)$ at $x_0$ . We want to use this polynomial to approximate $f(x)$ (Not only locally around $x_0$ !!!). Then what can we say about the $error(x)=f(x)-P_{n-1}(x)$ ?
- Taylor’s Theorem with Cauchy’s remainder (Global result): Let $n\geq1$ . Suppose $f^{(n-1)}$ exists & continuous on $[a,b]$ and $f^{(n)}$ exists on $(a,b)$ , and $x_0\in[a,b]$ . Then
  $f(x)=P_{n-1}(x)+\frac{f^{(n)}(c)}{n!}(x-x_0)^n$
  for some $c$ between $x$ and $x_0$ . We can interpret as a higher order MVT.
  - Proof
    Define $M=\frac{f(x)-P_{n-1}(x)}{(x-x_0)^n}$ , then we want to show $M=\frac{f^{(n)}(c)}{n!}$ .
    Define $g(x)=f(x)-P_{n-1}(x)-M(x-x_0)^n$ . Notice that
    $g^{(n)}(x)=f^{(n)}(x)-n! M$
    Hence we need to show there exists a $c$ between $x$ and $x_0$ such that $g^{(n)}(c)=0$ .
    通过 $n$ 次MVT, 在 $g(x)=0$ 到 $g^{(n)}(c)=0$ 之间搭桥。
    Also since $f(x)$ and $P_{n-1}(x)$ share the same first $n-1$ order derivatives, we have
    $g^{(k)}(x)=\frac{n!}{(n-k)!}M(x-x_0)^{n-k}, \forall k\leq n-1$
    Hence $g^{(k)}(x_0)=0,\forall k\leq n-1$ . Our choice of $M$ gives $g(x)=0$ .
    Apply MVT to $g$ on $[x_0,x]$ , $\exist x_1,s.t. \,\, g'(x_1)=\frac{g(x)-g(x_0)}{x-x_0}=0$ .
    Apply MVT to $g'$ on $[x_0,x_1],\exist x_2,s.t.\,\,g''(x_2)=\frac{g'(x_1)-g'(x_0)}{x_1-x_0}=0$ .
    …….
    Apply MVT on $g^{(n-1)}$ on $[x_0,x_{n-1}]$ , $\exists x_n,s.t.\,\,g^{(n)}(x_n)=0$ , as desired.
- Taylor’s Theorem with Peano’s remainder (Local result, usually used in quotients): Suppose $f^{(n)}(x_0)$ exists for some $n\geq1$ . Then
  $f(x)=f(x_0)+f'(x_0)(x-x_0)+...+\frac{f^{(n)}(x_0)}{n!}(x-x_0)^n+o((x-x_0)^n)$
  where $o((x-x_0)^n)$ represents any function such that $\lim_{x\to x_0}\frac{o((x-x_0)^n)}{(x-x_0)^n}=0$ .
  - Proof
    W.T.S. $\lim_{x\to x_0} \frac{f(x)-f(x_0)-f'(x_0)(x-x_0)-...-\frac{f^{n}(x_0)}{n!}(x-x_0)^n}{(x-x_0)^n}=0$
    At $x=x_0$ , the numerator is zero, the denominator is also zero, 0/0, use L’Hospital
    $=\lim_{x\to x_0}\frac{f'(x)-f'(x_0)-...-\frac{f^{(n)}(x_0)}{(n-1)!}(x-x_0)^{n-1}}{n(x-x_0)^{n-1}}$
    At $x=x_0$ , still 0/0, use L’Hospital again:
    $=\lim_{x\to x_0}\frac{f''(x)-f''(x_0)-...-\frac{f^{(n)}(x_0)}{(n-2)!}(x-x_0)^{n-2}}{n(n-1)(x-x_0)^{n-2}}$
    Repeat this process until we hit
    $=\lim_{x\to x_0}\frac{1}{n(n-1)...2} \frac{f^{(n-1)}(x)-f^{(n-1)}(x_0)-f^{(n)}(x_0)(x-x_0)}{x-x_0}$
    $=\lim_{x\to x_0} \frac{1}{n!}(\frac{f^{(n-1)}(x)-f^{(n-1)}(x_0)}{x-x_0}-f^{(n)}(x_0))=\frac{1}{n!}(f^{(n)}(x_0)-f^{(n)}(x_0))=0$
    Remark: we do not use L’Hospital to derive $=\lim_{x\to x_0}\frac{f^{(n)}(x)-f^{(n)}(x_0)}{n!}$ is because $f^{(n)}$ may not be continuos.
- (Hw10 Q5):
- (Hw10 Q6) Newton’s Method:
- (Hw10 Q7) Taylor’s Theorem for functions of several variables:
- (Hw10 Q8) Hessian matrix and second derivative test:

Inverse function Theorem
Let $\Omega$ be open in $\mathbb R^m$ and $f:\Omega\rarr \mathbb R^n$ be $C^1$ -smooth. Suppose $\exists a\in \Omega~s.t.~(f'(a))_{n\times n}$ is non-singular. (i) Then $\exists$ open sets $U,V\subset\mathbb R^n~s.t.~a\in U,b=f(a)\in V,$ $f$ is 1-1 on $U,\&~f(U)=V$ . (ii) Let $g(y)=(f|_{U})^{-1},y\in V$ , then $g$ is $C^1$ -smooth, $(g'(y))_{n\times n}=(f'(x))^{-1},\forall x\in U,y\in V$ . $f$ is said to be a (local) diffeomorphism from $U$ to $V$ .

Implicit Function Theorem
Let $\Omega$ be open in $\mathbb R^{m+n}=\mathbb R^{m}\times \mathbb R^{n}$ and $f:\Omega\rarr \mathbb R^n$ be $C^1$ -smooth. Write a generic point in $\mathbb R^{m+n}$ as $(x,y),x\in\mathbb R^{m},y\in \mathbb R^{n}$ . Suppose $\exists (a,b)\in \Omega~s.t.~f(a,b)=\vec 0$ , and $D_yf(x,y)|_{x=a,y=b}$ is non-singular. Then $\exists$ open sets $U\sub\mathbb R^{m},V\sub \mathbb R^{m+n}$ , s.t.
- $a\in U,(a,b)\in V$
- $\forall x\in U,\exists ! y\in \mathbb R^n~s.t.~(x,y)\in V~\&~f(x,y)=\vec 0$
Let $y=g(x),x\in U, g(x)$ is called the function implicitly defined by $f(x,y)=0$ . $g:U\rightarrow \mathbb R^n$ satisfies:
- $g(a)=b$
- $g$ is $C^1$ -smooth
- $f(x,g(x))=0,\forall x\in U$
- $(Dg(x))_{n\times m}=-(D_yf(x,y))^{-1}_{n\times n}D_xf(x,y)_{n\times m}),\forall x\in U$ .
[數學分析] Inverse Function Theorem
Chung-Han Hsieh's Blog
https://ch-hsieh.blogspot.com/2014/12/inverse-function-theorem.html
[數學分析] 隱函數定理
Chung-Han Hsieh's Blog
https://ch-hsieh.blogspot.com/2012/11/implicit-function-theorem.html
隐函数定理
22312;微积分中, 隐函数定理表明在适当的条件下 ...
https://www.bananaspace.org/wiki/%E9%9A%90%E5%87%BD%E6%95%B0%E5%AE%9A%E7%90%86

Back to index

11. Riemann Integral

Goal
- Given a set $S\subset R^n$ , how to rigorously define $\int_{S} f(x)dx$ .
- Necessary and sufficient conditions on $f(x)$ so that $\int_{S} f(x)dx$ exists.
- Generalization of Fubini’s Theorem (iterated integral).

Starting point: $S$ is a rectangle.
- Jargons
  - Let $Q$ be a closed rectangle in $R^n$ , then $Q=[a_1,b_1]\times...\times[a_n,b_n]=\{(x_1,...,x_n)\in R^n | a_1\leq x_1\leq b_1,...,a_n\leq x_n \leq b_n\}$ . and $|Q|=$ volume of $Q$ $\equiv (b_1-a_1)...(b_n-a_n)$ .
  - A partition of $[a,b]$ is a collection of cutting points $p=\{x_0,x_1,...,x_n\}$ . Intervals between two cutting points are the subintervals determined by $p$ .
  - Let $p_i$ be a partition of $[a_i,b_i]$ . We say n-tuple $(p_1,...,p_n)$ is a partition of $Q$ . $Q$ is divided into tiny sub-rectangles in the following manner: If $I_i$ is a subinterval determiend by $p_i$ , then $R=I_1\times ...\times I_2$ is called a sub-rectangle determined by $p$ , write as $R\leftarrow p$ . The norm of $p$ equals the largest size of all sub-rectangles determined by $p$ .
- (Riemann integrable): Let $f(x)$ be real-valued and bounded on $Q$ . We say $f$ is Riemann integrable on $Q$ if $\forall \varepsilon>0,\exist \delta>0,s.t.$ as long as $||p||<\delta$ , we have $|\sum_{R\leftarrow p} f(x_R)|R|-A|<\varepsilon$ , for some constant $A$ .
  - We write $\lim_{||p||\to 0} \sum_{R\leftarrow p} f(x_R)|R|=A$ .
  - Then $A$ is called the integral of $f$ on $Q$ , write as $\int_{Q} f(x)dx$ .
  - Agony: don’t know the value of $A$ $\Rightarrow$ Using limsup and liminf to squeezes $A$ .
- (Lower sum and upper sum): The choice of $x_R$ in each sub-rectangle is arbitrary. We can hence define the lower bound and upper bound of the Riemann sum as follows
  - $L(f;p)=\sum_{R\leftarrow p} (\inf _{x\in R} f(x))|R|\equiv \sum_{R\leftarrow p} m_{R}(f)|R|$
  - $U(f;p)=\sum_{R\leftarrow p} (\sup_{x\in R} f(x))|R|\equiv \sum_{R\leftarrow p} M_R(f)|R|$
  - Let $p$ and $p'$ be two partitions of $Q$ , define the common refinement at $p''=p\cup p'$ . Then we have $U(f;p)\geq U(f;p'')$ and $L(f;p)\leq L(f;p'')$ .
- (Lower integral and upper integral):
  - Upper integral: $\bar{\int_Q} f(x)dx=\inf_{p} U(f;p)$
  - Lower integral: $\underline{\int_{Q}} f(x)dx=\sup_{p} L(f;p)$
- Fact1 (Hw12 Q1): $f$ is R-integrable on $Q \Leftrightarrow \forall \varepsilon>0,\exist \delta>0,s.t.$ $0\leq U(f;p)-L(f;p)<\varepsilon$ whenever $||p||<\delta$ .
- Fact2: $f$ is R-integrable on $Q \Leftrightarrow \forall \varepsilon>0,\exist$ partition $p'$ such that $0\leq U(f;p')-L(f;p')<\varepsilon$ . (does not require $||p'||<\delta$ ).
- Fact3: Let $f$ be bounded on $Q$ . Then $f$ is R-integrable on $Q$ $\Leftrightarrow f$ is continuous almost everywhere on $Q.$
  - Intuition: Consider a sub-rectangle $R$ , if $f$ is continous on $R$ , then $f(x)$ is almost the same for $x$ within $R$ as long as $R$ is small enough, hence difference between the sup and inf is negligible. For those $R$ containing discontinuous points, the difference between the sup and inf may be large(but still bounded), but their area is negligible.
  - Basic measure theory (define continuous almost everywhere).
    A set $A \subset R^n$ is said to have “measure” zero (denoted as $|A|=0$ ) if $\forall \varepsilon>0$ , $\exist$ sequence $\{Q_i\}_{i=1}^n$ of closed rectangles in $R^n$ such that (i) $A\subset\cup_{i=1}^{\infty} \dot{Q}$ (ii) $\sum_{i=1}^{\infty} |Q_i|<\varepsilon$ .
    Basic facts: (i) If $B\subset A$ and $|A|=0$ , then $|B|=0$ . (ii) If $|A_i|=0,\forall i\geq 1$ , then $|\cup_{i=1}^{\infty} A_i|=0$ .
    Corollary (a countable set has measure zero) : If $A_i=\{a_i\}$ , then $|A_i|=0$ , and $|\cup_{i=1}^{\infty} \{a_i\}|=0$ . In particular, the set of rational numbers has measure zero.
  - We say $f(x)$ is continuous almost every where on $Q$ if the set of discountinous point has measure zero.
- (Fubini’s Theorem on $Q$ ): Let $A$ be closde rectangle in $R^n$ and $B$ cloased rectangle on $R^m$ . Let $Q=A\times B\subset R^{n+m}$ . Write generic point in $Q$ as $(x,y)$ , where $x\in A$ and $y\in B$ .Suppose $\int_{Q} f(x,y) d(x,y)$ exists and $\forall x\in A$ , $\int_{B} f(x,y)dy$ exists, then $\int_{A}(\int_{B}f(x,y)dy)dx$ exists and equals $\int_{A\times B} f(x,y)dxdy$ .

General $S$ , use closed rectangle $Q$ to cover

Properties of $\int_{S}f(x)dx$

@The Grand Inquisitor 哈哈小天，我在听最后两节课的录播，他说math research is much easier, you just need a pen and paper. 突然觉得做数学应该和做哲学很像吧。都是闭门造车。除非你做马克思那样的哲学哈哈哈。

@The Grand Inquisitor 天，最后一节课的内容是不是完全不会考

@Cindy Zeng 我刚在搞3040，还没开始看，最后一节你先别听吧，我当时听完倒数第二节感觉他已经把那三个问题回答的差不多了，我听完再告诉你有没有啥重点，不过听听课确实能找回一些rhythm

@The Grand Inquisitor 天，HW 13 P1 你作业里写的方法是绝对严谨的吧？感觉你和TA意思一样，就是在取 $x_R,y_R$ 的时候多用了一个消除中间的巧思……考试写你的方法应该木有问题吧？

@Cindy Zeng 嗯，他中间用那个 $\varepsilon-\delta$ 没有必要，直接用 $\lim_{|p|\to 0} \sum_{i=1}^n f(x_R)|R|$ 来算积分就行，只是这里的Riemann sum 搞出来加减相消后是 $f(x_n)x_n-f(x_0)x_0$ 和 $|p|$ 无关，稍微显得有点不自然。今天不知道咋回事，用notion又开始卡了😩

@The Grand Inquisitor 我也卡。而且/eq自动给我搞成block equation了不知道怎么调回inline

@Cindy Zeng 用shift+command+E可以直接快捷键打开inline equation，但整体打字延迟还是很高，尤其是中文

@The Grand Inquisitor 可能真的是文件太大了，有嗲担心他随时崩掉。另外我看了HW13最后一题，似乎倒数第二页写完也就只需要下个结论了？也许那张失效的图不是作业答案？

趁你睡觉的时候押题一波：

小题：

HW8P6 记得quiz 也出过。

大题（期中前*2）：

Qz3P1 Construct a bad sequence (of indices) from a NOT-Cauchy sequence.

大题（期中后*9）：

HW13P4 积分域交集零测则积分可加和。

HW13P5 利用continuity反证的经典，sharpened IVT.

HW12P2 Continuity证明，定义在闭区间上的连续函数零测的证明（这在原本的简单对角线的情况下不需要但是出一道变式，把平面变成空间，把f=1变成some continuous function就可以融合HW12P5，能考到很多东西。我预感他会有变式，后来他用的动词都不是“is”而是“related to” HW problems了。那道新题应该也是某种变化比较大的变式。

HW11P5 The first set of sufficient conditions that makes $f$ an open mapping: 1) $f$ is $C^1$ smooth, 2) Jacobian $f'(x)$ is non-singular $\forall x$ . “Extremely easy exercise, but a signicant result.”

HW8P5 NUS problem. “Try this yourself to see how much time you will take to come up with a correct and pretty proof!” It is pretty indeed.

次有可能考的：

HW12P1 挺综合的，把Riemann Integrability 原始定义的那些概念都考到了

@The Grand Inquisitor 天，Finally那段他到底在讲什么，我看了半天都没看懂。。this result 指代的是哪个？不需要 $(f^{-1})'$ non singular 吗。

@Cindy Zeng 我上午在高铁上，这边打起来依然很卡，我用overleaf给你押题好了